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Why  GAO  Did  This  Study 

VA  compensates  veterans  for  disabling 
conditions  incurred  in  or  aggravated  by 
military  service.  Veterans  can  appeal 
VBA’s  decisions  on  their  compensation 
claims,  first  to  VBA  and  then  to  the 
Board,  a  separate  agency  within  VA.  In 
fiscal  year  2015,  more  than  427,000 
appeals  were  pending  and  veterans 
waited  over  3  years  on  average  for 
decisions.  Of  this  total,  about  81 ,000 
were  pending  at  the  Board  and  the 
average  cumulative  time  veterans 
waited  for  a  decision  by  the  Board  in 
2015  was  almost  5  years. 

This  report  examines  VA’s  approaches 
to  address  challenges  it  identified  as 
contributing  to  lengthy  appeals 
processing  times,  and  the  extent  to 
which  those  approaches  are  consistent 
with  sound  planning  practices. 

GAO  focused  mainly  on  the  Board, 
which  experienced  an  increase  in 
workload  of  about  20  percent  from 
fiscal  year  2014  to  2015.  GAO 
reviewed  VA’s  proposed  plans  and 
actions  and  compared  them  to  sound 
practices  relevant  to  workforce 
planning  and  implementing  process 
redesign  and  new  information 
technology  identified  in  federal 
guidance,  such  as  internal  control 
standards,  and  prior  GAO  work.  GAO 
also  analyzed  VA’s  data  for  fiscal  years 
2011-2015  (the  most  recent  available) 
on  appeals  decision  timeliness  and 
workloads;  reviewed  relevant  federal 
laws,  regulations,  and  planning 
documents;  and  interviewed  VA 
officials  and  veterans  service 
organizations. 
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Additional  Planning  Would  Enhance  Efforts  to 
Improve  the  Timeliness  of  Appeals  Decisions 


What  GAO  Found 

The  Department  of  Veterans  Affairs’  (VA)  is  taking  steps  to  improve  the 
timeliness  of  its  benefit  compensation  appeals  process,  in  which  veterans  who 
are  dissatisfied  with  claims  decisions  by  the  Veterans  Benefits  Administration 
(VBA)  can  appeal  first  to  VBA,  and  then  to  the  Board  of  Veterans’  Appeals  (the 
Board).  VA  has  taken  actions  related  to  increasing  staff,  reforming  the  process, 
and  updating  information  technology  (IT),  which  are  consistent  with  relevant 
sound  planning  practices.  However,  gaps  in  planning  exist,  thereby  reducing  the 
agency’s  ability  to  ensure  that  these  actions  will  improve  the  timeliness  of 
disability  appeals  decisions. 

Increase  staff:  VA  determined  that  staff  resources  have  not  sufficiently  kept 
pace  with  increased  pending  appeals,  and  concluded  that  additional  staff  are 
needed,  particularly  at  the  Board,  to  improve  timeliness  and  reduce  its  appeals 
inventory.  The  Board  received  approval  to  hire  more  staff  in  fiscal  year  2017,  and 
expects  to  need  an  additional  hiring  surge  beginning  in  fiscal  year  2018.  As  of 
October  2016,  officials  estimated  that  if  the  agency  does  not  take  any  action, 
such  as  increasing  staff  in  2018,  veterans  may  have  to  wait  an  average  of  8.5 
years  by  fiscal  year  2026  to  have  their  appeals  resolved. 

Consistent  with  sound  workforce  planning  practices,  VA  modeled  different 
options  for  increasing  staff  levels  to  support  its  conclusion  that  staff  increases  in 
conjunction  with  process  change  would  reduce  the  appeals  inventory  sooner. 
However,  contrary  to  sound  practices,  VA  often  used  fixed  estimates  for  key 
variables  in  its  models — such  as  staff  productivity — rather  than  a  range  of 
estimates  (sensitivity  analysis)  to  understand  the  effect  variation  in  these  key 
variables  could  have  on  staffing  needs.  Also,  VA’s  written  workforce  plans — 
which  cover  recruiting,  hiring  and  training — did  not  include  detailed  steps,  time 
frames,  and  mitigation  strategies  consistent  with  sound  workforce  planning 
practices.  For  example,  while  VA  has  established  a  center  for  excellence  in 
hiring  to  focus  on  recruitment  and  hiring,  the  agency  has  not  finalized  training  or 
telework  plans  or  otherwise  mitigated  space  constraints  that  it  encountered  for 
hiring  staff  in  fiscal  year  2017.  Without  a  timely,  detailed  workforce  plan,  VA  risks 
delays  in  hiring  and  preparing  staff  to  help  manage  workloads  as  soon  as 
possible. 

Reform  process:  VA  determined  that  new  evidence — which  a  veteran  can 
submit  at  any  point  during  his  or  her  appeal — inefficiently  causes  an  additional 
round  of  reviews,  and  thus  delays  appeals  decisions,  and  in  response  it 
proposed  legislation  (not  enacted)  to  streamline  the  process.  Consistent  with 
sound  practices  for  process  redesign,  VA  worked  with  veterans  service 
organizations  (VSO)  and  other  key  stakeholders  in  developing  the  proposal,  and 
continued  to  update  VSOs  about  the  development  of  its  implementation  plans. 
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What  GAO  Recommends 

GAO  is  making  five  recommendations 
to  VA  and  one  matter  for 
congressional  consideration.  VA 
should:  apply  sensitivity  analyses 
when  projecting  staff  needs,  develop  a 
more  timely  and  detailed  workforce 
plan,  develop  a  robust  plan  for 
monitoring  process  reform,  develop  a 
strategy  for  assessing  process  reform, 
and  create  a  schedule  for  IT 
improvements  that  takes  into  account 
plans  for  potential  process  reform.  VA 
concurred  in  principle  with  the  five 
recommendations,  but  believes  it  has 
met  the  intent  of  those 
recommendations  and  does  not  need 
to  take  additional  action.  GAO 
disagrees  and — while  recognizing 
VA’s  ongoing  efforts — believes  further 
action  is  needed  on  all  five 
recommendations  to  improve  VA’s 
ability  to  successfully  implement 
reforms,  as  discussed  in  the  report. 

VA  disagreed  with  an  additional  draft 
recommendation  that  it  incorporate 
pilot  testing  of  its  proposed  appeals 
process  into  implementation  plans  and 
pursue  necessary  legislative  authority. 
VA  cited  its  perspective  that  the 
appeals  process  is  broken  and  that 
piloting  a  new  process  would  result  in 
further  delays  to  veterans  appealing 
their  disability  decisions.  GAO 
maintains  that  the  benefits  of  pilot 
testing — which  provides  an  opportunity 
to  resolve  implementation  challenges 
and  make  refinements  to  the  process 
on  a  smaller  scale — outweigh  the 
potentially  negative  consequences  of 
delaying  full  implementation. 

Therefore,  GAO  removed  the 
recommendation  and  added  a  matter 
for  congressional  consideration  stating 
that  Congress  should  consider 
requiring  that  appeals  process  reform 
be  subject  to  a  pilot  test. 


VA’s  proposed  reform  is  promising,  but  there  are  several  gaps  in  its 
implementation  plans.  In  particular,  VA  plans  to  fully  implement  appeals  process 
reform  at  the  Board  as  well  as  at  VBA  regional  offices  across  the  country  while  it 
concurrently  manages  the  existing  appeals  inventory,  a  hiring  surge,  and  planned 
system  changes  discussed  below.  However,  VA’s  plans  run  counter  to  sound 
redesign  practices  that  suggest  pilot  testing  the  process  changes  in  a  more 
limited  fashion  before  full  implementation,  in  order  to  manage  risks  and  help 
ensure  successful  implementation  of  significant  institutional  change.  VA  officials 
told  GAO  that  pilot  testing — which  would  require  legislation  to  implement — will 
prolong  a  process  that  is  fundamentally  broken  and  delay  urgently  needed 
repairs.  However,  without  pilot  testing  VA  may  experience  challenges  and 
setbacks  on  a  broader  scale,  which  could  undermine  planned  efficiencies  and 
other  intended  outcomes.  In  addition,  VA  has  not  sufficiently  identified  how  it  will 
monitor  progress,  evaluate  efficiency  and  effectiveness,  identify  trouble  spots, 
and  otherwise  know  whether  implementation  of  its  proposed  process  change  is 
on  track  and  meeting  expectations.  The  absence  of  a  robust  monitoring  plan  with 
success  criteria  is  inconsistent  with  sound  planning  practices  for  redesign  and 
places  the  agency  at  risk  of  not  being  able  to  quickly  identify  and  address 
setbacks.  In  addition,  the  timeliness  measures  that  VA  currently  plans  to  report  to 
Congress  and  the  public  lack  transparency  because  they  focus  on  individual 
parts  of  the  agency  and  pieces  of  the  new  process  rather  than  overall  appeals 
resolution  time  from  the  veterans’  perspective.  Without  a  strategy  for  assessing 
the  proposed  new  process  that  includes  comprehensive  measures,  VA,  the 
public,  and  Congress  cannot  know  the  extent  to  which  the  proposed  process 
represents  an  improvement  over  the  old  process. 

Update  technology:  VA  determined  that  the  computer  system  supporting  its 
appeals  process  is  outdated,  prone  to  failures,  and  does  not  adequately  support 
electronic  claims  processing.  VA  proposed  a  new  IT  system  to  reduce  delays  in 
appeals  to  the  Board,  and  better  integrate  data  from  other  systems.  Consistent 
with  sound  practices,  VA  clearly  laid  out  the  scope  and  purpose  of  IT  upgrades, 
and  identified  risks  and  strategies  to  mitigate  them.  However,  the  agency’s  plan 
lacks  details  for  how  and  when  its  new  system  will  be  implemented,  as 
suggested  by  sound  planning  practices  for  implementing  new  technology. 

Without  a  detailed  schedule,  VA  risks  not  having  new  systems  aligned  with 
potential  changes  in  the  appeals  process  when  they  are  implemented. 
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Abbreviations 


AOJ 

Agency  of  Original  Jurisdiction 

DOD 

Department  of  Defense 

DSVA 

Digital  Service  at  VA 

FTE 

full-time  equivalent 

IT 

information  technology 

NOD 

Notice  of  Disagreement 

SOC 

Statement  of  the  Case 

SSOC 

Supplemental  Statement  of  the  Case 

VA 

Department  of  Veterans  Affairs 

VACOLS 

Veterans  Appeals  Control  and  Locator  System 

VBA 

Veterans  Benefits  Administration 

VBMS 

Veterans  Benefits  Management  System 

VLJ 

Veterans  Law  Judge 

VSO 

Veterans  Service  Organization 
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March  23,  2017 
Congressional  Addressees 

In  fiscal  year  2015,  the  Department  of  Veterans  Affairs  (VA)  provided 
about  $63.7  billion  in  disability  compensation  payments  to  about  4.1 
million  veterans  with  disabling  conditions  that  were  incurred  during  or 
aggravated  by  their  military  service.  VA’s  Veterans  Benefits 
Administration  (VBA)  determines  veterans’  eligibility  and  payment 
amounts  based  on  medical  records  and  examinations,  military  service 
records,  and  other  evidence.  A  veteran  dissatisfied  with  VBA’s  initial  claim 
decision  may  appeal  at  two  VA  levels — first  to  VBA,  then  to  the  Board  of 
Veterans’  Appeals  (Board).  In  recent  years,  VA  has  reduced  its  high 
inventory  of  initial  disability  claims  to  fewer  than  100,000,  but  at  the  same 
time,  appeals  inventories  have  grown.  In  fiscal  year  2015,  according  to 
VA  data,  the  number  of  appeals  pending  at  VBA  and  the  Board  was  more 
than  427,000  and  veterans  waited  over  3  years  on  average  for  an 
appeals  decision  made  in  that  fiscal  year  by  either  VBA  or  the  Board.  VA 
reported  that  appeals  workloads  at  the  Board  almost  doubled  from  41,000 
in  fiscal  year  201 1  to  81 ,000  in  fiscal  year  201 5,  increasing  about  20 
percent  from  fiscal  year  2014  to  fiscal  year  2015  and  the  average 
cumulative  time  veterans  waited  for  an  appeals  decision  by  the  Board  in 
2015  was  almost  5  years  (1,789  days).  GAO  has  designated  VA  disability 
compensation  and  other  federal  disability  programs  as  a  high-risk  area, 
due  in  part  to  the  challenges  VA  faces  in  deciding  disability  benefit  claims 
(including  appeals)  in  a  timely  manner.1 

In  light  of  VA’s  growing  appeals  workload,  and  delays  in  deciding 
appeals,  we  conducted  this  review  under  the  Comptroller  General’s 
authority  to  conduct  evaluations  of  significant  current  issues  on  his  own 
initiative.  In  this  report,  we  examined:  (1)  the  key  challenges  VA  identified 
that  affect  the  timeliness  of  appeals  decided  by  the  Board,  and  what  VA  is 
doing  to  address  them;  and  (2)  the  extent  to  which  VA’s  proposed 
approaches  to  address  these  challenges  are  consistent  with  sound 
planning  practices.  Our  review  largely  focused  on  the  Board’s  role  in  the 
disability  compensation  appeals  process,  given  recent  increases  in  its 


^AO,  High-Risk  Series:  Progress  on  Many  High  Risk  Areas,  While  Substantial  Efforts 
Needed  on  Others,  GAO-17-317  (Washington,  D.C.:  Feb.  15,  2017). 
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workloads,  and  that  GAO  has  not  conducted  a  review  of  the  Board  and  its 
handling  of  appeals  in  over  10  years.2 

To  examine  key  timeliness  challenges  and  VA’s  efforts  to  address  them, 
we  reviewed  relevant  agency  reports,  planning  documents,  budget 
submissions,  and  federal  laws  and  regulations.  We  also  interviewed  VA 
officials  representing  the  Board,  VBA,  and  the  Office  of  Information  and 
Technology.  From  these  sources,  we  identified  three  approaches  that  VA 
stressed  as  important  to  achieving  significant  improvements  in  appeals 
resolution  timeliness:  (1)  increasing  staff;  (2)  streamlining  the  appeals 
process;  and  (3)  improving  information  technology  (IT)  support. 

To  provide  additional  perspective  on  workload  challenges,  we  obtained 
VA  administrative  data  for  fiscal  years  201 1  through  2015  on  pending 
appeals  inventory,  age  of  appeals  determined  in  each  year,  number  of 
staff,  quality  measures  (such  as  error  rates  and  the  types  of  errors 
specific  to  appeals),  and  the  extent  to  which  veterans  appeal  cases 
repeatedly.  We  selected  a  5-year  span  for  our  analysis  using  data  for  the 
most  recent  years  available  at  the  time  of  our  request.3  We  validated  data 
provided  by  VA  by  comparing  what  we  received  against  a  data  extract  we 
collected  from  the  Board’s  Veterans  Appeals  Control  and  Locator  System 
(VACOLS) — which  is  used  by  VA  to  track  the  status  of  appeals.4  We 
determined  that  these  data  were  sufficiently  reliable  for  our  purposes.  To 
provide  further  insight  on  workload  challenges,  we  also  interviewed 
representatives  of  four  veterans  service  organizations  (VSO) — the 


2While  the  Board  handles  appeals  of  benefit  decisions  across  VA,  we  chose  to  focus  on 
disability  compensation-related  appeals  because  they  represent  the  vast  majority  of  the 
Board’s  workload.  In  fiscal  year  2015,  about  94  percent  of  all  Board  decisions  were  on 
appeals  of  VBA  decisions  on  veterans’  disability  compensation  claims. 

3Within  these  5  years,  VA  changed  some  of  the  measures  it  uses  to  track  and  publicly 
report  appeals  inventories  and  timeliness.  Specifically,  in  fiscal  year  2015,  VA  stopped 
publicly  reporting  Appeals  Resolution  Time  data — time  from  the  date  the  veteran  files  a 
notice  of  disagreement  until  a  final  decision  is  made.  Instead,  VA  began  tracking  and 
reporting  Board  timeliness  based  on  when  the  claim  was  certified  by  VBA  as  an  appeal 
ready  for  review  by  the  Board.  VA  was  able  to  provide  us  with  data  from  their  older 
methods  of  tracking  appeals  inventories  and  timeliness  for  our  data  comparison  across 
multiple  years. 

4ln  all  but  two  cases,  we  were  able  to  arrive  at  figures  that  were  within  5  percent  of  those 
provided  by  VA.  We  could  not  duplicate  VA’s  figures  for  the  total  time  needed  to  reach  a 
final  decision  on  appeals  and  the  end  of  year  claims  inventory  numbers  because  they 
were  based  on  point-in-time  data  that  cannot  be  replicated  from  the  VACOLS  data  we 
received.  In  these  two  cases  we  worked  with  VA  to  obtain  explanations  as  to  how  it 
calculated  these  figures  and  we  found  its  responses  to  be  satisfactory. 
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American  Legion,  Disabled  American  Veterans,  National  Veterans  Legal 
Services  Program,  and  Paralyzed  Veterans  of  America — each  of  which 
has  staff  members  who  represent  veterans  before  the  Board,  and  had 
participated  in  recent  VA  efforts  to  improve  the  appeals  process. 

To  assess  the  extent  to  which  VA’s  approaches  to  address  challenges 
are  consistent  with  sound  planning  practices,  we  identified  best  practices 
and  other  criteria  through  a  review  of  relevant  literature,  such  as 
government-wide  internal  control  standards,5  and  prior  GAO  reports.6  We 
specifically  selected  prior  GAO  reports  where  we  defined  a  number  of 
desirable  characteristics  of  an  effective,  results-oriented  plan,  or 
components  of  sound  planning  practices.7  We  also  reviewed  additional 
guidance  pertinent  to  process  change  and  project  management.8 
Although  there  is  no  established  set  of  requirements  for  all  plans,  we 
incorporated  relevant  best  practices  and  criteria  into  a  data  collection 
instrument  reflecting  sound  planning  practices  and  used  this  instrument  to 
guide  our  evaluation  of  each  of  VA’s  approaches.  We  further  sought  out 
and  reviewed  sources  identifying  key  practices  and  desirable 
characteristics  related  to  VA’s  three  key  approaches  (staffing,  process 
reform,  and  IT  upgrades)  to  improving  review  processes  and  workloads 
including  strategic  planning  practices,  transformation  plans,  technology 
upgrades,  and  human  capital  improvements  (mostly  increasing  staffing). 
For  example,  for  approaches  related  to  staffing,  we  referred  to  reports 
with  key  principles  for  effective  strategic  workforce  planning;9  for  process 


5OMB.  Circular  A-1 1  (Washington,  D.C.:  July  2016). 

6GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO-14-704G 
(Washington,  D.C.:  Sep.  10,  2014). 

7GAO,  Combating  Terrorism:  Evaluation  of  Selected  Characteristics  in  National  Strategies 
Related  to  Terrorism,  GAO-04-408T  (Washington,  D.C.:  Feb.  3,  2004),  and  Executive 
Guide:  Effectively  Implementing  the  Government  Performance  and  Results  Act, 
GAO/GGD-96-1 1 8  (Washington,  D.C.:  June  1996). 

8Simon  Blackburn,  Sarah  Ryerson,  Leigh  Weiss,  Sarah  Wilson,  and  Carter  Wood,  Insights 
into  Organization:  How  Do  I  Implement  Complex  Change  at  Scale?  Dallas,  TX:  McKinsey 
&  Company,  May  201 1 ,  and  Project  Management  Institute,  Inc.  A  Guide  to  The  Project 
Management  Body  of  Knowledge  (PMBOK  Guide)  (Newtown  Square,  PA.:  2004). 

9GAO,  Human  Capital:  Key  Principles  for  Effective  Strategic  Workforce  Planning. 
GAO-04-39  (Washington,  D.C.:  Dec.  11, 2003),  and  GAO,  Human  Capital:  Strategies  to 
Help  Agencies  Meet  Their  Missions  in  an  Era  of  Highly  Constrained  Resources. 
GAO-14-168  (Washington,  D.C.:  May  7,  2014). 
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reform,  we  used  GAO  guidance  on  business  process  reengineering10  (or 
redesign);  and  for  IT  upgrades,  we  referred  to  GAO  guidance  and  reports 
on  information  technology  planning.* 11  For  a  list  of  the  sources  of  sound 
practices  and  criteria  we  used  to  evaluate  VA’s  approaches,  see  related 
products  at  the  end  of  this  report. 

We  also  reviewed  VA’s  analysis  assessing  the  effectiveness  of  two  of  the 
proposed  approaches  (staffing  increases  and  process  reform)  for 
addressing  workload  challenges.  Specifically,  we  reviewed  data  analysis 
spreadsheets  and  related  documentation  (such  as  historical  data  and 
assumptions),  conducted  interviews,  and  observed  a  demonstration  of 
how  VA  compiled  the  data  for  its  projections.  We  did  not  assess  the 
feasibility  of  the  options  compared  in  its  analysis. 

We  conducted  this  performance  audit  from  February  2016  to  March  2017 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Background 


VA  Disability 
Compensation  Benefit 
Claims  Process 


VA’s  process  for  deciding  veterans’  eligibility  for  disability  compensation 
begins  when  a  veteran  submits  a  claim  to  VA.  The  claim  is  reviewed  at 
one  of  VBA’s  56  regional  offices  where  staff  members  assist  the  veteran 
by  gathering  any  additional  evidence,  such  as  military  and  medical 
records,  needed  to  evaluate  the  claim.12  Based  on  this  evidence,  and  the 
results  of  any  necessary  medical  examinations,  VBA  decides  whether  the 
veteran  is  entitled  to  compensation  and,  if  so,  how  much.  VBA  assigns  a 


10GAO,  Business  Process  Reengineering  Assessment  Guide — Version  3, 
GAO/AIMD-IO.1.15  (Washington,  D.C.:  May  1997).  In  this  report,  we  refer  to 
reengineering  as  appeals  process  “redesign.” 

11  GAO,  Information  Technology  Investment  Management:  A  Framework  for  Assessing 
and  Improving  Process  Maturity,  GAO-04-394G  (Washington,  D.C.:  March  2004). 

12VA  has  a  statutory  “duty  to  assist”  the  veteran  by  gathering  evidence,  such  as  military 
and  medical  records,  needed  to  make  the  initial  claim  decision.  This  requirement  also 
applies  to  appeals,  as  well  as  initial  claims. 
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rating  of  0  to  100  percent  disability  in  increments  of  10  percentage  points 
depending  on  the  severity  of  the  disability.  This  rating  percentage  then 
determines  the  monthly  payment  amount  the  veteran  will  receive.13 
According  to  VA  data,  in  many  cases  (74  percent),  the  veteran  submitting 
a  claim  either  is  already  a  beneficiary  but  is  seeking  increased 
compensation,  or  the  veteran  was  denied  benefits  previously  and  is 
claiming  them  again.  In  fiscal  year  2015,  VBA  decided  1.4  million 
compensation  claims  and  had  an  inventory  of  363,000  claims  at  the  end 
of  the  fiscal  year.  As  previously  noted,  in  fiscal  year  2015,  VA  paid  about 
$63.7  billion  in  disability  compensation  to  about  4.1  million  veterans. 


VA  Appeals  Process  A  veteran  dissatisfied  with  VBA’s  initial  claim  decision  can  generally 

appeal  within  one  year  from  the  date  of  VBA’s  notification  letter  to  the 
veteran.  According  to  the  Board,  veterans  appeal  most  often  because 
they  believe  VBA:  (1)  incorrectly  denied  them  compensation  for  service- 
connected  disabilities,  or  (2)  under-rated  their  service-connected 
disabilities.  An  appeal  begins  with  the  veteran  filing  a  Notice  of 
Disagreement  (NOD).  VBA  then  re-examines  the  case  and  generally 
issues  a  Statement  of  the  Case  (SOC)  that  represents  its  decision. 

A  veteran  who  is  or  remains  dissatisfied  with  VBA’s  decision  can  file  an 
appeal  with  the  Board.14  In  filing  that  appeal,  the  veteran  could  indicate 
whether  they  would  like  a  Board  hearing.  VBA  prepares  the  claim  file  for 
Board  review  and  certifies  it  as  ready  for  review.  If  the  veteran  requests  a 
hearing  so  they  can  present  new  evidence  or  arguments,  the  Board  will 
generally  hold  a  hearing  either  by  video  conference  or  at  a  local  VBA 


1 3 

A  veteran  with  a  30-percent  or  higher  rating  receives  a  larger  amount  based  on  the 
number  of  dependents.  Also,  VBA  provides  a  larger  monthly  payment  to  a  veteran  who 
lost  certain  organs  or  one  or  more  limbs,  or  who  is  found  in  need  of  aid  and  attendance. 
Further,  a  veteran  with  a  rating  of  at  least  60  percent — or  a  lesser  percent,  in  certain 
circumstances — may  be  paid  a  Total  Disability  Individual  Unemployability  benefit, 
equivalent  to  payment  for  a  1 00-percent  rating,  if  the  veteran  is  found  unable  to  obtain  or 
maintain  gainful  employment  due  to  service-connected  disabilities. 

14 

According  to  VA  officials,  in  filing  that  appeal  or  at  any  point  during  the  appeals  process, 
the  veteran  may  submit  new  evidence,  and  VA  has  a  statutory  duty  to  assist  the  claimant 
in  obtaining  evidence  in  support  of  the  appeal  throughout  the  entire  appeals  process. 
Depending  on  several  factors,  the  new  evidence  might  not  necessitate  the  issuance  of  a 
Supplemental  Statement  of  the  Case  (SSOC).  If  the  appeal  was  received  on  or  after 
February  2,  201 3,  VBA  would  only  issue  a  SSOC  if  the  veteran  or  their  representative 
requests  a  review  in  writing,  according  to  VA  officials. 
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regional  office.15  The  Board  reports  to  the  Office  of  the  Secretary  of 
Veterans  Affairs,  and  is  independent  of  VBA.  The  Board’s  members,  also 
known  as  Veterans  Law  Judges  (VLJ),  decide  appeals  and  are  supported 
by  attorneys  and  administrative  staff.  After  the  appeal  is  docketed  at  the 
Board,  a  VLJ  or  panel  of  VLJs  reviews  the  evidence  and  either  (1)  grants 
the  claimed  benefit,  (2)  denies  the  benefit,  or  (3)  returns  (remands)  the 
claim  to  VBA  for  additional  work  on  one  or  more  issues  pertinent  to  the 
claim  and  a  new  decision.  According  to  VA,  the  Board  remands  an  appeal 
to  VBA  in  cases  where  consideration  of  new  evidence,  clarification  of 
evidence,  correction  of  procedural  defect,  or  any  other  action  it  deems  is 
essential  to  achieve  a  proper  decision.  If  the  veteran  is  unsatisfied  with 
the  Board’s  final  decision,  the  veteran  can  continue  an  appeal  beyond  VA 
to  federal  court.  Such  an  appeal  begins  with  the  U.S.  Court  of  Appeals  for 
Veterans  Claims,  then  may  go  to  the  U.S.  Court  of  Appeals  for  the 
Federal  Circuit,  and  finally  to  the  U.S.  Supreme  Court.  See  figure  1  for  a 
representation  of  the  appeals  process  for  VA  disability  compensation 
benefit  decisions. 


15The  vast  majority  of  Board  hearings  (96  percent  in  fiscal  year  201 5)  are  held  by  video 
conference  or  at  a  regional  office,  after  which  the  appeal  is  docketed  at  the  Board.  For  the 
small  percent  of  hearings  that  are  held  in  person  at  the  Board’s  central  office  in 
Washington  D.C. —  4  percent  in  fiscal  2015  and  6  percent  in  fiscal  year  201 6,  according  to 
VA — the  appeal  is  docketed  prior  to  the  hearing. 
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Figure  1:  Current  Process  for  Appeals  of  Veterans  Affairs’  (VA)  Disability  Compensation  Benefit  Decisions 


Average  time 
elapsed  for 
appeals  decided 
by  the  Board 
in  FY20153 


419  days 

from  receipt 
of  NOD  to 
issuing  SOC 


537  days 

from  receipt 
of  appeal  to 
certifying  it 

222  days 

from  certifying 
appeal  to  it 
being  docketed 


270  days 

from  appeal 
docketed  to 
Board  decision 


Source:  U.S.  Department  of  Veterans  Affairs  (VA)  data  and  documents.  |  GAO-17-234 

aThis  timeline  does  not  include  time  from  when  VA  issues  the  Statement  of  the  Case  to  receipt  of  the 
veteran’s  formal  request  to  have  their  appeal  reviewed  by  the  Board  and  the  average  remand  time, 
which  for  appeals  decided  by  the  Board  in  fiscal  year  2015  took  39  and  255  days  respectively.  These 
numbers  do  not  add  up  to  the  1 ,789  days  previously  noted  in  part  due  to  averaging  and  VA  pulled  the 
fiscal  year  2015  data  from  its  system  at  different  times. 

bUnder  the  traditional  review  option,  the  veteran  may  present  new  evidence  and  receive  a  formal 
hearing.  In  general,  the  reviewer  can  change  VBA’s  original  decision  based  only  on  new  evidence,  or 
a  clear  and  unmistakable  error  in  the  original  decision.  Alternatively,  the  veteran  may  elect  a  review 
by  a  Decision  Review  Officer,  who  reviews  the  record  without  deference  to  VBA’s  original  decision, 
and  can  revise  that  decision  based  on  a  difference  of  opinion.  If  needed,  the  Decision  Review  Officer 
may  also  pursue  additional  evidence  or  discuss  the  appeal  informally  with  the  veteran  or  the  veteran’s 
representative. 
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cAn  appeal  to  the  Board  must  be  filed  within  60  days  of  the  SOC  being  mailed,  or  within  the 
remainder  of  the  1-year  appeal  period  from  the  date  of  mailing  the  decision  notification. 

dAccording  to  VA  officials,  when  new  evidence  is  submitted  at  any  point  prior  to  filing  of  an  appeal  to 
the  Board,  the  office  that  decided  the  original  claim  (Agency  of  Original  Jurisdiction  or  AOJ)  is 
required  to  review  the  evidence  in  the  first  instance,  undertake  any  further  development  warranted, 
and  issue  a  SSOC  addressing  the  newly  submitted  evidence,  if  a  SOC  was  already  issued.  For  new 
evidence  submitted  by  the  veteran  or  his  or  her  representative  with  or  after  the  appeal  is  filed  with  the 
Board,  the  AOJ  is  not  required  to  issue  a  SSOC  unless:  the  appeal  is  filed  with  the  Board  before 
February  2,  2013,  or  (for  appeals  filed  with  the  Board  on  or  after  February  2,  2013)  the  veteran  or  his 
or  her  representative  requests  in  writing  that  the  AOJ  first  review  such  evidence. 

eThe  vast  majority  of  Board  hearings  (96  percent  in  fiscal  year  2015)  were  held  by  video  conference 
or  at  a  regional  office,  which  occur  before  the  appeal  is  docketed  at  the  Board.  Hearings  held  at  the 
Board’s  central  office  in  Washington  D.C. — 4  percent  in  fiscal  year  2015 — take  place  after  the  appeal 
is  docketed. 


Inventories  and  Timeliness  According  to  VA  officials,  the  number  of  appeals  filed  has  increased 
Of  VA  Appeals  steadily  as  has  the  length  of  time  needed  for  the  agency  to  make  a  final 

decision.  At  the  end  of  fiscal  year  2015,  according  to  VA  data,  VA  had 
over  427,000  pending  appeals,  approximately  81 ,000  of  which  were  at 
the  Board.  While  the  percentage  of  pending  appeals  awaiting  decisions 
from  the  Board  was  less  than  a  quarter  of  all  pending  appeals,  the  fiscal 
year  201 5  inventory  was  almost  double  the  41 ,000  pending  at  the  end  of 
fiscal  year  201 1 .  About  20  percent  of  this  growth  occurred  from  fiscal  year 

2014  through  201 5. 16 

According  to  Board  data,  timeliness  has  worsened  since  fiscal  year  201 1 
as  well.  From  fiscal  years  201 1  through  2015,  the  average  amount  of  time 
needed  for  the  Board  to  make  a  final  decision  once  the  appeal  is 
docketed  increased  from  240  to  270  days.  In  addition,  the  proportion  of 
cases  taking  the  longest  to  resolve  (from  when  the  Board  receives  the 
certified  appeal  to  them  making  a  final  decision) — over  600  days — 
increased  from  10  percent  in  fiscal  year  2011  to  14  percent  in  fiscal  year 

2015  (see  fig.  2).  Given  that  the  median  time  for  the  Board  to  decide  an 
appeal  was  145  days  in  fiscal  year  2015  (compared  to  an  average  of  270 
days),  these  data  suggest  that  a  relatively  small  number  of  appeals  is 
driving  up  the  Board’s  reported  average  processing  times.  To  illustrate, 

VA  officials  noted  one  case  where  a  veteran  appealed  27  times  over  the 
course  of  25  years  before  the  original  appeal  was  concluded.17 


16According  to  our  analysis  of  data  from  the  Board  of  Veterans’  Appeals,  the  Board  issued 
about  56,000  decisions  and  also  received  almost  70,000  appeals  in  fiscal  year  2015. 

17According  to  VA  officials,  one  issue  related  to  this  appeal  is  still  pending  with  the  Board; 
therefore,  as  of  February  2017,  this  appeal  had  been  pending  for  over  30  years. 
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Figure  2:  Board  of  Veterans’  Appeals  Decision  Times,  from  the  Board’s  Receipt  of 
the  Certified  Appeal  to  Its  Decision,  Fiscal  Years  2011-2015 


Total  number  of  Board  decisions 


60,000 

50,000 
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4  Percentage  that  took  more  than  600  days 
4  401  to  6 00  days 
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4  200  or  fewer  days 


Fiscal  year 

Source:  GAO  analysis  of  Board  of  Veterans' Appeals  data.  |  GAO-17-234 


VA  Has  Proposed 
Increased  Staffing, 
Process  Reform,  and 
Updated  IT  to 
Improve  Appeals 
Timeliness,  and  Has 
Taken  Action  for  All 
Three  Approaches 


VA  has  identified  three  broad  approaches  for  addressing  factors  that  it 
identified  as  having  contributed  to  increased  appeal  inventories  and 
reduced  timeliness  of  appeals  decisions,  and  has  already  taken  action  on 
all  three  fronts.  Citing  staffing  levels  that  have  not  kept  pace  with 
workloads,  VA  secured  additional  Board  staff  for  fiscal  year  2017,  and 
analyzed  options  for  another  hiring  surge  in  fiscal  2018.  Concerned  that 
its  appeals  process  contributes  to  delays  in  appeals  decisions — because 
new  evidence  may  be  submitted  at  any  juncture  and  because  VA  may  be 
continually  required  to  develop  or  obtain  additional  evidence — VA 
developed  a  legislative  proposal  for  streamlining  its  appeals  process, 
including  new  appeals  options  designed  to  accelerate  decision-making. 
Finally,  VA  has  put  forth  plans  to  modernize  its  current,  outdated,  and 
inefficient  computer  system. 


VA  Proposed  Hiring  More 
Staff  to  Address 
Increasing  Workloads 


VA  has  proposed  increasing  staff  at  the  Board,  as  well  as  VBA,  to 
manage  its  increasing  inventory  of  appeals  and  to  address  related 
declines  in  the  timeliness  of  appeals  resolutions.  VA  officials  stated  that 
there  is  a  direct  and  proportional  correlation  between  the  number  of 
employees  and  the  number  of  final  appeals  decisions,  and  that  Board 
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workloads  especially  have  increased  faster  than  the  number  of 
employees  staffed  to  the  Board.  Specifically,  officials  have  concluded  that 
staff  resources  within  the  Board  have  not  been  sufficient  to  adjudicate  the 
increasing  number  of  appeals,  ultimately  lengthening  appeals  resolution 
times.  According  to  VA,  in  fiscal  year  2015  increases  in  staff  (VLJs, 
attorneys,  and  support  staff),  as  represented  by  full-time  equivalents 
(FTEs),  allowed  the  Board  to  make  the  highest  number  of  decisions  in 
nearly  30  years.  However,  despite  Board  staff  increasing  by  21  percent 
from  fiscal  years  201 1  through  2015,  officials  said  that  this  increase  was 
not  sufficient  to  address  the  growing  inventory  of  pending  appeals,  which 
doubled  during  the  same  time  period  (see  fig.  3). 


Figure  3:  Pending  Appeals  Inventory  and  Staff  (Full-Time  Equivalents)  at  the  Board  of  Veterans’  Appeals,  Fiscal  Years  2011- 
2015 


Inventory  of  pending  appeals 


Number  at  end  of  year 

81,022 


Fiscal  year 

Source:  GAO  analysis  of  Board  of  Veterans' Appeals  data.  |  GAO-17-234 
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Although  the  increase  in  Board  staff  brought  about  a  record  number  of 
appeals  decisions  in  fiscal  year  2015,  according  to  VA  data  we  reviewed, 
each  appeal  took  an  average  of  about  3  months  (97  days)  longer  to  reach 
a  final  decision  than  in  fiscal  year  2012. 18  Similarly,  in  fiscal  year  2015 

1  ft 

VA  did  not  provide  data  for  fiscal  year  201 1  on  appeals  resolution  time,  which  was  the 
year  used  in  other  comparisons  throughout  this  report. 
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one  Board  FTE  produced  an  average  of  86  appeals  decisions,  down  from 
91  completed  per  FTE  in  fiscal  year  201 1 .  Growing  workloads  and  the 
increased  complexity  of  cases,  according  to  Board  officials,  have 
contributed  to  these  longer  appeal  resolution  times.  More  specifically, 
officials  said  that  claims  have  become  more  complicated  due  to  not  only 
the  number  and  complexity  of  injuries  and  illnesses,  but  also  to  advances 
in  medicine  that  have  improved  survival  rates  from  catastrophic  injuries, 
experienced  by  today’s  veterans.  VA  officials  estimated  that  if  the  number 
of  FTEs  and  number  of  appeals  decided  per  FTE  stays  steady  or 
decreases,  appeals  resolution  times  will  continue  to  lengthen. 

Specifically,  as  of  October  2016,  VA  projected  that  if  nothing  else 
changes,  and  if  the  number  of  FTEs  hold  steady  at  the  fiscal  year  201 7 
number  (922  FTEs  for  the  Board  and  1 ,495  for  VBA),  the  inventory  of 
appeals  could  exceed  1  million  in  fiscal  year  2026,  which  would  mean  that 
veterans  would  wait  an  average  of  8.5  years  for  a  final  appeals  decision.19 

In  light  of  this  assessment,  VA  concluded  that  increasing  the  number  of 
FTEs  at  the  Board  is  a  key  step  in  mitigating  the  current  and  future 
pending  inventory  of  appeals  and  ultimately  improving  appeals  decision 
timeliness.  In  2016,  VA  set  a  goal  to  decide  the  vast  majority  (90  percent) 
of  appeals  (including  both  those  reviewed  by  VBA  and  the  Board)  within  1 
year  by  2021 .  As  an  initial  step  toward  this  goal,  VA  requested  and 
received  a  funding  amount  that  the  agency  asserted  would  allow  it  to  fund 
an  additional  242  FTEs  for  the  Board  in  fiscal  year  201 7  (a  36  percent 
increase  over  the  680  FTEs  funded  in  fiscal  year  2016)  for  a  total  of  922 
FTEs.  VA  also  concluded,  however,  that  this  increase  in  staff  will  not  be 
enough  to  reduce  its  appeals  workload  and  decrease  appeals  processing 
time.  Therefore,  VA  estimated  the  need  for  a  subsequent  hiring  surge  of 
up  to  1 ,458  FTEs  beginning  in  fiscal  year  201 8  to  reduce  the  current 
pending  appeals  inventory.20 

To  understand  the  need  for  and  implications  of  a  future  hiring  surge,  VA 
modeled  different  staffing  scenarios.  Initially,  VA  compared  how 
increasing  staff  in  combination  with  and  without  proposed  changes  to  the 


19This  projection  combines  final  decisions  made  by  VBA  as  well  as  those  made  by  the 
Board,  and  includes  a  number  of  underlying  assumptions  that  are  discussed  later  in  this 
report.  Such  assumptions  include,  for  example,  that  the  number  of  initial  claims  filed 
(increase  by  1  percent  per  year)  and  appeal  rates  hold  steady  at  1 1  percent,  consistent 
with  historical  averages  since  1996. 

on 

According  to  agency  officials,  VA  plans  to  continue  to  update  and  revise  its  model,  which 
may  change  its  projections. 
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appeals  process  would  achieve  inventory  reductions,  and  at  what  cost. 
VA  determined  that  by  combining  staff  increases  with  a  new  process,  it 
could  clear  pending  appeals  faster  and  at  a  lower  cost  than  if  it  hired 
additional  staff  under  the  current  process.21  In  response  to  congressional 
inquiries,  in  September  2016  VA  also  modeled  the  cost  and  impact  on 
appeals  inventories  of  four  surge  options  beginning  in  fiscal  year  201 8  (in 
addition  to  planned  hiring  in  fiscal  year  201 7). 22  VA  estimated,  for 
example,  that  projected  pending  appeals  in  fiscal  year  2017  (535,726) 
would  be  cleared  in  10  years  under  option  2,  compared  to  a  60  percent 
reduction  over  the  same  time  period  if  there  were  no  hiring  surge.  See 
table  1  for  a  comparison  of  the  four  options. 


Table  1:  Veteran  Affairs’  (VA)  Hiring  Surge  Options — Represented  in  Full  Time  Equivalents  (FTE) — Estimated  Impact  on 

Pending  Appeals  Inventory 

Option 

Additional  FTEs  across  VA 
(beyond  those  approved  for 
fiscal  year  2017) 

Estimated  impact  on  projected 

VA  pending  appeals  in  fiscal  year 
(FY)  2017  (535,726) 

Baseline 

0 

Reduced  by  about  60  percent  in  10  years  (FY  2026) 

1 

244 

Reduced  by  about  76  percent  in  10  years  (FY  2026) 

2 

584 

Cleared  in  10  years  (FY  2026) 

3 

943 

Cleared  in  8  years  (FY  2024) 

4 

1,458 

Cleared  in  6  years  (FY  2022) 

Source:  VAdata.  I  GAO-17-234 


Note:  The  estimated  impact  on  the  projected  pending  appeals  is  based  on  VA  also  reforming  the 
current  appeals  process. 


VA  has  proposed  changes  to  its  appeals  process  to  address  causes  of 
delays  in  resolving  appeals.  The  key  challenge  VA  identified  was  the 
open-ended  nature  of  its  disability  appeals  process,  whereby  a  veteran 
can  submit  additional  evidence  numerous  times  at  any  point  during  the 
VA  appeals  process,  which  can  cause  another  cycle  of  re-adjudication. 
Specifically,  when  a  veteran  submits  additional  pertinent  evidence  after 
VA’s  initial  decision  on  their  claim,  VA  is  generally  required  to  review  the 


VA  Proposed  Reforming 
Current  Appeals  Process 
to  Address  Factors 
Contributing  to  Delays  in 
Resolving  Appeals 


21  Based  on  analysis  conducted  at  that  time,  VA  concluded  it  could  clear  pending  appeals 
in  5  years  at  an  estimated  cost  of  $700  million  over  baseline  spending  for  the  staff  surge 
combined  with  proposed  process  changes,  compared  to  10  years  and  $2.4  billion  over 
baseline  for  staff  surge  under  the  current  process. 

22Based  on  its  modeling,  VA  expected  that  the  number  of  staff  would  eventually  decrease 
after  fiscal  year  201 9  as  a  result  of  attrition,  commensurate  with  projections  of  a  declining 
appeals  inventory. 
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evidence,  develop  any  other  needed  evidence,  and  issue  another 
decision.  This  is  the  case  regardless  of  whether  the  veteran  submits  the 
additional  evidence  to  VBA  or  to  the  Board  and,  for  appeals  pending 
before  the  Board,  the  submission  of  additional  evidence  may  result  in  a 
remand  to  VBA  for  further  development.  VA  reported  that  in  fiscal  year 
2015,  the  Board  remanded  about  46  percent  of  appeals  to  VBA  for 
additional  development.23  Of  those  remanded  appeals,  which  may  involve 
more  than  one  issue,  VA  reported  that  about  60  percent  of  the  reasons 
those  appeals  were  returned  to  VBA  were  due  to  the  open  record  that 
allows  veterans  to  introduce  new  evidence  at  any  point  during  the 
appeal.24  VA  reported  that  in  fiscal  year  2015,  it  took  VBA  an  additional 
255  days  on  average  to  complete  remand  development  and  for  the 
appeal  to  be  re-docketed  at  the  Board.  VA  also  reported  that  in  fiscal  year 
2015  it  took  the  Board  an  average  of  244  additional  days  to  complete  its 
subsequent  review  of  the  returned  remands  and  decide  the  appeal. 
According  to  VA,  this  re-adjudication  can  occur  multiple  times  and  can 
add  years  to  the  time  needed  to  reach  a  final  decision  on  an  appeal. 

Board  and  VSO  officials  also  identified  factors  within  VBA’s  initial  claim 
process — and  outside  of  the  Board’s  control — that  cause  delays  in 
veterans  receiving  final  decisions  on  their  appeal.  Specifically, 

•  According  to  Board  and  VSO  officials,  VBA’s  decision  notification 
letters  are  unclear  and  confusing.  In  particular,  the  officials  stated  that 
these  letters  do  not  adequately  explain  why  claims  were  denied  and 
do  not  clearly  identify  the  evidence  a  veteran  needs  to  provide  to  fully 
support  a  claim  on  appeal.  As  a  result,  some  veterans  may  appeal 
unnecessarily,  or  they  may  appeal  without  providing  the  evidence 
needed  to  support  their  claims. 


no 

According  to  VA  officials,  the  46  percent  remand  rate  is  based  on  the  Board’s  historical 
method  for  reporting  appeals,  which  counted  the  disposition  of  appeals  in  a  given  year  as 
an  allowance,  remand,  denial,  or  dismissal.  Officials  noted  that  this  method  did  not 
capture  every  remand  because  it  did  not  reflect  appeals  involving  multiple  issues  with 
different  dispositions.  For  example,  VA  counted  an  appeal  as  an  allowance  if  one  issue 
was  allowed  and  one  remanded.  According  to  VA  officials,  29  percent  of  appeals  decided 
in  fiscal  year  2015  had  been  remanded  to  VBA  at  some  point  during  the  appeals  process 
for  additional  development. 

24Board  officials  stated  that  they  track  remands  by  reason  and  by  issue.  A  remanded  case 
may  involve  multiple  issues  remanded  for  multiple  reasons.  For  example,  the  veteran  may 
have  appealed  VBA’s  initial  decision  on  compensation  for  more  than  one  disabling 
condition,  and  the  Board  may  have  different  reasons  for  remanding  each  issue. 
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•  VSO  officials  we  interviewed  said  that  some  delays  are  attributable  to 
errors  in  VBA’s  initial  decisions.  They  suggested  that  errors  may  have 
occurred  because  VBA  rushed  some  decisions  in  its  initiative  to 
reduce  its  backlog  of  claims  pending  more  than  125  days.  Such  errors 
can  lead  to  Board  remands  and  VBA  re-work.  For  example,  the  Board 
may  remand  an  appeal  because  VBA  failed  to  meet  the  “duty  to 
assist”  responsibilities  to  a  veteran.25  According  to  the  Board,  41 
percent  of  the  reasons  for  the  remands  in  fiscal  year  2015  were  due  to 
a  VBA  error. 

•  Board  and  VSO  officials  also  cited  delays  in  VBA’s  transmittal  of 
appeals  to  the  Board  as  a  possible  cause  for  the  delays  in  Board 
decisions.  When  a  veteran  files  an  application  with  VBA  to  appeal  to 
the  Board,  VBA  prepares  the  case  file  for  transfer  to  the  Board, 
certifies  that  the  case  file  is  complete  and  ready  for  Board  review,  and 
transmits  the  file  to  the  Board.  According  to  VA  data  on  appeals 
decisions  made  by  the  Board  in  fiscal  year  2015,  it  took  on  average 
537  days  to  process  the  appeal  from  receipt  to  certification.  Docketing 
appeals  that  had  been  certified  to  the  Board  added  an  average  of  222 
days  to  processing  times  for  appeals  decisions  made  in  fiscal  year 
2015.  VSOs  (two  of  the  four  we  interviewed)  told  us  that  they  noticed 
these  delays  occurring  as  VA’s  focus  shifted  to  clearing  the 
compensation  benefit  claims  backlog. 

To  address  process-related  challenges,  VA’s  approach  has  been  to 
develop  a  proposal  to  streamline  the  appeals  process  and  to  ask  the 
Congress  to  make  changes  in  the  laws  governing  the  process.  In  April 
2016,  VA  issued  a  draft  summary  of  a  proposed  streamlined  appeals 
process  that  reflected  collaboration  with  its  stakeholders.26  This  summary 
was  accompanied  by  draft  legislation  for  the  Congress’  consideration.  If 
enacted  into  law,  the  draft  legislation  would  make  process  changes  that 


25According  to  VA  officials,  under  VA’s  new  framework,  the  “duty  to  assist”  requirement 
would  not  apply  following  VBA’s  initial  decision  on  a  claim  or  supplemental  claim,  although 
the  Board  could  remand  a  case  to  correct  duty  to  assist  errors  that  occurred  prior  to  the 
initial  decision  being  reviewed. 

26VA,  A  New  Framework  for  Veterans  Claims  and  Appeals,  April  22,  2016.  According  to 
VA,  it  does  not  plan  to  produce  a  final  version. 
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VA  identified  as  needed  to  streamline  the  appeals  process.27  According  to 
VA,  key  to  the  proposed  process  changes  would  be  replacing  the  current 
appeals  process,  which  begins  in  VBA,  with  a  process  giving  a  veteran 
four  options — two  in  VBA  and  two  in  the  Board.28  As  presented  in  VA’s 
framework,  these  options  would  be: 

•  Ask  VBA  to  review  its  initial  decision  based  on  the  same  evidence. 
Under  this  option,  the  veteran  would  not  be  able  to  submit  new 
evidence  or  request  a  VBA  hearing,  and  would  not  be  subject  to  VA’s 
“duty  to  assist”  requirement.  A  VBA  official  (at  a  level  higher  than  the 
official  who  made  the  initial  decision)  would  review  the  record 
supporting  the  initial  decision,  and  issue  a  new  decision. 

•  The  veteran  could  file  a  “supplemental  claim”  with  VBA,  asking  VBA  to 
review  its  initial  decision,  while  providing  additional  evidence.  Under 
this  option,  the  veteran  could  also  request  a  VBA  hearing.  Another 
VBA  official  (at  the  same  level  as  the  original  VBA  decision-maker) 
would  review  the  revised  record,  including  the  additional  evidence 
from  the  veteran,  and  issue  a  new  decision. 

•  The  veteran  could  file  a  Notice  of  Disagreement  directly  with  the 
Board,  bypassing  a  VBA  review.  The  veteran  would  have  two  options 
with  the  Board: 

•  Ask  the  Board  to  review  only  the  existing  record  without  a  hearing 
and  then  issue  a  decision. 


27ln  the  1 14th  Congress,  various  bills  that  would  have  changed  the  appeals  process  were 
introduced  in  the  Congress.  This  included  a  bill  (H.R.  5620)  passed  by  the  House  of 
Representatives  on  September  14,  2016.  That  bill,  and  other  Senate-introduced  bills  (such 
as  S.  3170  and  S.  3328),  had  been  pending  in  the  Senate  Committee  on  Veterans’  Affairs 
during  the  1 14th  Congress,  but  were  not  enacted.  At  the  time  our  review  was  complete, 
additional  bills  that  would  change  the  appeals  process  had  been  introduced  in  the  1 1 5th 
Congress  (e.g.  H.R.  457,  H.R.  611  and  S.  152). 

28ln  its  appeals  process  reform  proposal,  VA  refers  to  appeals  in  two  ways,  depending  on 
which  VA  unit  reviews  the  appeal.  VA  refers  to  appeals  within  VBA  as  “reviews;”  and 
considers  them  extensions  of  VBA’s  initial  claims  process.  Meanwhile,  VA  generally  refers 
to  appeals  directly  to  the  Board  as  “appeals,”  and  considers  the  Board  to  be  VA’s  appeals 
agency.  For  the  purposes  of  this  report,  we  refer  to  all  formal  veterans’  disagreements 
with  VBA’s  initial  decision  as  “appeals.” 
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•  Alternatively,  ask  the  Board  to  review  additional  evidence,  conduct 
a  hearing  before  issuing  its  decision,  or  both.29 

See  figure  4  for  a  representation  of  the  options  in  VA’s  proposed 
simplified  appeals  process. 


Figure  4:  Department  of  Veterans  Affairs  (VA)  Proposed  Appeals  Process  and  VA’s  Estimated  Breakdown  of  Veterans’  Use 
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Source:  VBA’s  proposed  framework.  |  GAO-17-234 
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VA  officials  anticipate  that  the  proposed  appeals  process  described 
above  will  expedite  appeals  in  a  number  of  ways,  most  notably: 

•  For  those  appeals  where  no  additional  evidence  is  submitted,  and  no 
formal  hearing  is  conducted  (indicated  as  “VBA  conducts  local  higher- 
level  review”  and  “Board  reviews  record  without  new  evidence  or  a 
hearing”  in  figure  4),  the  re-review  of  the  original  record  could 
expedite  a  final  appeals  decision.  In  addition,  VA’s  “duty  to  assist” 
requirements  would  only  apply  to  VBA  for  initial  and  supplemental 
claims.  Unlike  the  current  process,  in  which  the  Board  may  remand 
appeals  to  VBA  to  consider  new  evidence,  the  Board  would  only 
remand  appeals  under  the  new  process  in  cases  in  which  the  Board 
found  that  VBA  failed,  in  its  initial  or  supplemental  claim  processing,  to 
meet  VA’s  “duty  to  assist”  the  veteran.  VA  estimates  that,  once  the 


pq 

For  the  purpose  of  managing  its  caseload,  the  Board  would  have  two  dockets  (in 
addition  to  legacy  cases):  (1 )  for  appeals  with  additional  evidence,  which  may  or  may  not 
involve  a  hearing,  and  (2)  for  appeals  with  a  review  of  the  existing  record  without 
additional  evidence  ora  hearing. 


Page  16 


GAO-17-234  VA  Disability  Benefits 


new  process  is  fully  implemented,  remands  will  steadily  decrease  and 
eventually  occur  in  as  few  as  5  percent  of  appeals.30 

•  When  the  veteran  appeals  directly  to  the  Board,  VBA  would  no  longer 
be  required  to  review  the  record  (including  any  additional  evidence), 
prepare  statements  of  its  findings  (i.e.,  prepare  SOCs  or  SSOCs),  and 
certify  appeals  as  ready  for  Board  review. 

VA  has  estimated  that  as  a  result  of  these  process  changes — in 
combination  with  increased  FTEs — the  Board  could  complete  cases 
faster,  deciding  many  more  appeals  per  FTE  in  fiscal  year  2018 
compared  to  fiscal  year  2015.  More  specifically,  VA  estimated  that  the 
Board  could  complete  an  average  of  180  appeals  decisions  per  FTE 
without  a  hearing  and  130  with  a  hearing,  compared  to  the  average  of  86 
total  decisions  per  FTE  in  fiscal  year  2015.  We  discuss  VA’s  estimates  in 
more  detail  later  in  this  report. 

While  VA’s  proposal  reflects  VA’s  intent  to  expedite  appeals  resolutions,  it 
also  contains  various  protections  for  veterans  that  are  intended  to 
address  stakeholders’  concerns  about  fairness.  Notably,  such  protections 
include  the  following: 

•  In  contrast  to  the  “one  size  fits  all”  process,  the  proposed  reform 
allows  the  veteran  to  choose  an  option  that  best  fits  the  circumstances 
of  a  veteran’s  claim.  As  shown  in  figure  2  above,  a  veteran  could 
choose  to  have  VBA  review  the  initial  decision  or  they  could  appeal 
directly  to  the  Board.  Also,  the  veteran  would  have  the  option  to  have 
either  VBA  or  the  Board  review  the  existing  record,  without  having  to 
submit  new  evidence  and/or  request  a  formal  hearing.  VA  expects 
that  these  options  could  help  the  veteran  obtain  a  faster  decision  from 
VBA  or  the  Board.  Per  VA’s  framework,  under  the  new  process,  the 
veteran  would  have  up  to  1  year  from  VBA’s  initial  decision  to  choose 
an  option.  Further,  if  the  veteran  is  unsuccessful  in  one  appeal  option, 
the  veteran  could,  within  1  year,  choose  another  option.31  However, 


on 

VA  estimates  this  improvement  over  the  41  percent  error  rate  in  fiscal  year  201 5  will  be 
due  to  a  reduction  in  the  number  of  steps  where  such  errors  can  occur  and  improved 
feedback  loops  between  the  Board  and  the  VBA  office  that  made  the  initial  decision. 

CM 

As  previously  noted,  currently,  a  veteran  generally  has  1  year  from  the  date  of  VBA’s 
notification  letter  to  file  a  Notice  of  Disagreement  and  begin  the  appeals  process.  The 
veteran  can  file  an  appeal  with  the  Board  within  60  days  after  VBA  mails  a  SOC  or  within 
the  remainder  of  the  1-year  appeal  period  from  the  date  of  mailing  the  decision 
notification. 
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according  to  VA,  an  appeal  for  a  higher-level  review  by  VBA  without 
new  evidence  cannot  directly  follow  a  Board  decision. 

•  VBA  would  be  required  to  provide  more  information  in  letters  notifying 
veterans  of  decisions  involving  a  denial  of  benefits,  which  could  help 
veterans  make  more  informed  decisions  on  whether  to  appeal,  which 
option  to  pursue,  and  what  additional  evidence  (if  any)  to  provide.32 
The  inclusion  of  additions  to  such  notifications  in  VA’s  proposed 
legislation  addresses  stakeholders’  concerns  that  veterans  did  not 
have  enough  information  to  decide  whether  they  should  appeal,  or 
what  additional  evidence  they  needed  to  provide,  thereby  resulting  in 
unnecessary  appeals  or  delays  in  appeals. 

•  A  veteran  who  is  not  fully  satisfied  with  the  result  of  any  lane  would 
have  1  year  to  seek  further  review,  while  preserving  an  effective  date 
for  benefits  based  on  the  date  the  veteran  filed  the  original  claim  with 
VBA.  This  would  help  ensure  a  veteran  is  not  penalized  for  pursuing 
an  appeal  to  the  Board.  For  example,  under  VA’s  proposal,  a  veteran 
denied  benefits  by  the  Board  could  choose  to  have  VBA  conduct 
another  review,  by  filing  a  supplemental  claim  with  additional 
evidence.  In  contrast,  under  current  law,  if  a  veteran  appeals  to  the 
Board  and  is  denied  (and  does  not  appeal  to  a  federal  court),  the 
veteran  must  generally  reopen  the  claim,  or  start  over,  by  filing 
another  claim  with  VBA.  If  the  veteran  is  subsequently  granted 
benefits,  the  benefits  would  generally  be  awarded  from  the  date  on 
which  the  new  (not  original)  claim  was  filed,  which  could  result  in  the 
veteran  not  receiving  retroactive  compensation  payments. 


VA  Has  Plans  to 
Modernize  Its  Outdated 
Appeals  IT  System 


VA  has  plans  to  modernize  its  current  IT  system,  which  it  determined  is 
antiquated  and  a  source  of  delays  in  processing  appeals.  VA  currently 
uses  the  Veterans  Appeals  Control  and  Locator  System  (VACOLS)  to 
track  and  manage  its  appeals  workload.  VA  identified  a  number  of 
reasons  why  it  believes  VACOLS  should  be  replaced,  including  that: 


32Under  current  law,  VBA’s  notification  must  explain  the  procedures  for  obtaining  a  review 
of  its  decision.  Also,  if  VBA  denies  benefits  to  the  veteran,  its  notification  must  state  the 
reasons  for  its  decision,  and  the  evidence  it  considered.  Under  VA’s  proposal,  when  VBA 
denies  benefits,  its  notification  must  include  information  on:  (1)  the  issues  adjudicated,  (2) 
the  evidence  considered,  (3)  the  applicable  laws  and  regulations,  (4)  findings  favorable  to 
the  veteran,  (5)  findings  as  to  which  element(s)  were  found  not  to  have  been  satisfied 
leading  to  the  denial  of  the  claim,  (6)  how  the  veteran  can  access  the  evidence  used  in 
making  the  decision,  and  (7)  the  criteria  the  veteran  must  satisfy  for  the  claim  to  be 
granted. 
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•  The  system  is  based  on  outdated  technology  dating  from  the  1990s 
that  VA  determined  would  be  difficult  to  modify  to  meet  the  changing 
needs  of  VA. 

•  VA  designed  VACOLS  around  a  paper-based  VA  claims  process  and 
as  a  result,  VACOLS  does  not  adequately  support  a  fully  electronic 
environment.  According  to  VA,  although  VACOLS  has  been  patched 
to  some  extent  to  handle  paperless  appeals,  the  Board  relies  on 
paper  briefs  to  help  manage  its  appeals  workflow. 

•  VACOLS’s  lack  of  automation,  integration  with  other  VA  systems,  and 
error  checks  results  in  mistakes  and  lost  productivity.  According  to 
VA,  individual  employees  spend  a  significant  amount  of  time 
correcting  data  entry  errors  that  would  be  avoided  if  cases  were 
automatically  transferred  to  the  Board.  For  instance,  they  said  that 
after  cases  are  transferred  to  the  Board,  a  team  of  employees  must 
manually  review  and  correct  most  incoming  cases  due  to  issues  with 
labeling,  mismatched  dates,  and  missing  files.  Via  an  internal  study, 
VA  determined  that  up  to  88  percent  of  cases  transferred  to  the  Board 
had  such  errors.  Additionally,  VA  notes  that  data  entry  errors  can 
result  in  instances  where  paperless  cases  are  mislabeled  as  paper- 
based.  These  cases  will  not  show  up  as  certified  in  VACOLS  and  the 
Board  will  erroneously  wait  for  a  paper  case  that  will  never  arrive. 

•  VACOLS  is  central  to  appeals  processing,  thus  a  system  outage 
would  halt  the  processing  of  appeals  across  VA,  either  paper  or 
electronic,  until  VACOLS  is  repaired,  according  to  VA. 

VA  expects  its  VACOLS  replacement  to  improve  the  efficiency  of  its 
appeals  decisions.  Its  planned  replacement — called  Caseflow — is 
intended  to  address  the  limitations  of  VACOLS  and  better  support 
processing  appeals  in  a  paperless  environment.  According  to  VA, 
Caseflow  is  being  developed  in  an  agile  process  in  which  new  functions 
are  added  to  the  system  as  they  are  completed.  In  fiscal  year  2016,  VA 
developed  two  initial  deliverables.  According  to  VA,  the  first  is  intended  to 
automate  and  introduce  consistency  to  the  process  of  transferring 
appeals  to  the  Board.  The  second  introduced  the  ability  for  staff  to  access 
documentation  from  the  Veterans  Benefits  Management  System 
(VBMS) — VA’s  system  for  processing  claims — which  VA  believes  will 
eventually  allow  users  to  review  appeals  more  efficiently.  As  of  February 
2017,  VA  officials  also  noted  the  agency  is  in  the  process  of  developing 
additional  components,  including  document  review  software  for  VLJs  and 
attorneys,  and  a  component  to  better  track  appeals  that  are  remanded  to 
VBA.  According  to  officials,  VA’s  longer  term  plans  include  a  broad 
roadmap  for  continuously  adding  improvements  to  Caseflow.  For 
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instance,  VA  has  plans  to  build  into  the  system  the  capability  to  generate 
performance  metrics,  using  a  component  called  Caseflow  Dashboard.  VA 
states  that  the  dashboard  will  be  able  to  draw  on  various  VA  data 
systems  and  provide  information  on  bottlenecks  in  the  appeals  process, 
quantify  improvements  in  the  appeals  process — including  those 
attributable  to  improved  IT  systems — and  track  the  reasons  for  and 
number  of  remands.  While  Caseflow  improvements  are  being  made,  VA 
reported  it  plans  to  maintain  VACOLS  as  a  redundant  resource  until  the 
new  system  is  fully  complete,  at  which  point  VACOLS  will  be  retired. 


VAApplied  Several 
Sound  Planning 
Practices  in  Its  Efforts 
to  Improve  Its 
Appeals  Processing 
Timeliness,  but  Fell 
Short  in  Other  Areas 


VA  Weighed  Several 
Factors  When  Determining 
Its  Need  for  More  Staff, 
but  Did  Not  Fully  Assess 
Risks  or  Document  Its 
Workforce  Plans 


VA  acted  consistently  with  sound  planning  practices  in  determining  its 
need  for  additional  staff,  but  it  did  not  fully  consider  risks  and 
uncertainties  in  its  approach.  Sound  practices  for  effective  planning 
suggest  that  agencies  should  consider  alternative  solutions  to  a  problem; 
assess  the  risk  of  unintended  consequences;  and  use  data  to  analyze  the 
problem,  including  unknowns.33  Consistent  with  these  concepts  and  more 
specific  sound  workforce  planning  practices,  VA  considered  various  hiring 
options,  such  as  hiring  staff  under  the  current  versus  VA  proposed 
process,  and  modeling  appeals  inventories  under  four  hiring  surge 
options.34  VA  considered  a  number  of  factors  when  comparing  the  four 
hiring  options  including  historical  data  on  the  volume  and  complexity  of 
appeals,  estimates  of  future  growth  in  appeals,  and  the  productivity  of 
employees  in  estimating  the  number  of  Board  staff  needed  to  meet  its 
timeliness  goals.  For  instance,  the  Board  reviewed  past  data  on  the 


oo 

For  key  characteristics  of  sound  planning  of  any  government  project,  see  GAO-04-408T. 
34See  GAO-04-39  and  GAO-14-168. 
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productivity  of  new  staff — which  is  generally  lower  for  a  period  of  time 
until  individuals  acclimate  to  their  jobs — and  factored  this  into  the 
modeling  assumptions  used  to  project  the  number  of  Board  staff  needed. 

More  specifically,  sound  workforce  planning  practices  suggest  that 
agencies  identify  the  resources  needed  to  manage  the  risks  of 
implementing  new  processes  and  conduct  scenario  planning  to  determine 
these  needs.35  While  VA  considered  a  number  of  factors  when  analyzing 
hiring  options,  it  initially  made  many  assumptions  using  a  single  set  of 
estimates  instead  of  using  a  sensitivity  analysis  to  consider  a  range  of 
estimates.36  These  assumptions  could  have  significant  implications  for 
how  accurately  VA  identifies  needed  resources.  For  example,  in  its 
scenario  analysis  VA  assumed:  (1 )  that  an  average  of  50  percent  of  those 
veterans  appealing  will  refile  their  appeal  and  go  through  two  of  the  four 
appeals  process  options  before  being  satisfied;  and  (2)  that  the  Board  will 
be  able  to  decide  130  appeals  per  FTE,  and  do  so  within  3  years  (1,095 
days),  for  appeals  with  hearings  and  decide  180  appeals  per  FTE  within  1 
year  for  appeals  without  hearings.  Because  the  Board  did  not  consider 
alternate  sets  of  assumptions,  VA  does  not  know  the  potential  effect  that 
variations  in  these  key  variables  could  have  on  staffing  needs. 

In  response  to  discussions  with  us  on  its  scenario  analyses,  VA  recently 
conducted  further  analyses  using  alternative  estimates  for  key  factors, 
although  the  agency’s  analyses  fell  short  of  the  previously  discussed 
sound  practices  for  estimating  outcomes  based  on  assumptions. 
Specifically,  VA  calculated  the  effect  on  appeals  inventories  and 
timeliness  if  VA  decided  20  percent  fewer  appeals,  if  VA  decided  more 
claims  and  thus  had  more  appeals  than  expected,  and  if  the  breakdown 
of  options  that  veterans  selected  for  their  appeals  review  is  different  than 
the  50/50  split  VA  projected.  The  20  percent  reduction  in  productivity 
alone  could  add  2.5  years  to  VA’s  estimate  of  how  long  it  would  take  to 
clear  the  appeals  inventory  under  hiring  surge  option  4.  However,  VA  ran 
a  sensitivity  analysis  for  only  one  of  the  four  hiring  surge  options  and  did 
not  analyze  the  compounded  effect  of  different  assumptions  together.  By 


35GAO,  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
managing  Capital  Program  Costs,  GAO-09-3SP  (Washington  D.C.:  March  2009).  See  also 
GAO-04-39. 

36Sensitivity  analysis,  used  in  scenario  planning,  is  an  analysis  to  determine  how  sensitive 
outcomes  are  to  changes  in  the  assumptions.  The  assumptions  that  deserve  the  most 
attention  should  depend  on  the  dominant  benefit  and  cost  elements  and  the  areas  of 
greatest  uncertainty  of  the  program  or  process  being  analyzed.  See  GAO-09-3SP. 
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not  comprehensively  conducting  sensitivity  analyses,  VA  is  hampered  in 
its  ability  to  anticipate  and  plan  for  different  contingencies,  and  risks  being 
caught  off  guard  and  potentially  hiring  an  inappropriate  number  of  staff. 
Hiring  too  few  staff  could  result  in  it  taking  longer  to  reduce  the  inventory 
of  pending  appeals,  while  hiring  too  many  staff  could  result  in  higher 
expenditures  than  needed  and  exacerbate  other  challenges,  such  as 
ensuring  sufficient  office  space,  training,  and  other  supports  for  newly 
hired  staff,  as  discussed  below.  VA  has  acknowledged  that  some  of  its 
assumptions,  and  thus  projections,  are  based  on  unknowns  and  that  it  will 
need  to  continuously  rerun  the  models  with  updated  data. 

VA  also  identified  strategies  and  resources  needed  for  recruiting,  hiring, 
and  training  staff  in  fiscal  year  2017;  however,  aspects  of  VA’s  workforce 
planning  fall  short  of  sound  workforce  planning  practices  that  suggest 
having  timely  written  plans  with  a  systematic  approach  and  detailed 
steps,  time  frames,  and  mitigation  strategies  to  help  identify  where 
resources  and  investments  should  be  targeted.37  As  noted  below,  VA  has 
identified  strategies  and  taken  some  positive  steps  related  to  recruiting, 
hiring,  and  training  staff  in  fiscal  year  2017,  although  these  plans 
sometimes  lacked  certain  details  specifically  covered  in  sound  workforce 
planning  principles  in  time  to  inform  ongoing  efforts. 

•  Recruitment  and  Hiring:  Consistent  with  sound  workforce  planning 
practices,  officials  have  worked  to  develop  a  center  for  excellence  in 
hiring  to  coordinate  workforce  planning  and  develop  strategies  for 
recruiting  and  hiring  staff  quickly.  However,  the  center  was 
established  in  the  last  quarter  of  fiscal  year  2016  to  support  hiring 
beginning  early  in  fiscal  year  2017.  To  date,  the  Board  developed  a 
project  to  recruit  recent  law  school  graduates  and  alumni  in  fiscal  year 
2017,  according  to  Board  officials.  It  also  has  formed  a  committee  of 
over  90  volunteers  to  develop  recruitment  materials,  identify 
opportunities,  and  make  contact  with  law  schools;  developed  a 
PowerPoint  presentation  for  the  visits;  and  conducted  a  few  initial 
presentations  at  law  schools.  However,  the  agency  had  not  yet 
worked  out  specific  goals  such  as  the  number  of  presentations  or 
resulting  applications,  average  time  taken  to  recruit,  and  skills  needed 
in  recruits,  or  identified  metrics  (other  than  hiring  goals)  against  which 
it  would  measure  the  effectiveness  of  the  recruitment  efforts.38 


37See  GAO-14-168. 

oo 

GAO.  Human  Capital:  A  Self-Assessment  Checklist  for  Agency  Leaders. 
GAO/OCG-OQ-14G  (Washington,  D.C.:  September  2000). 
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Also  consistent  with  sound  workforce  planning  practices,  Board 
officials  told  us  that  they  considered  lessons  learned  from  a  2013 
hiring  surge,  although  the  agency  did  not  provide  documentation  of 
these  lessons  learned.39  Having  established  a  goal  of  hiring  25-52 
new  employees  per  month  from  October  2016  through  April  2017,  the 
Board  subsequently  faced  challenges  finding  space  for  staff  coming 
aboard  in  fiscal  year  201 7. 40  Specifically,  as  of  October  2016,  the 
Board  was  reconfiguring  its  office  space  to  accommodate  the  planned 
242  new  FTEs  in  fiscal  year  2017,  employing  nearly  all  of  its 
conference  rooms,  and  developing  a  plan  for  using  telework  and  office 
sharing  to  accommodate  staff  until  the  space  is  available  for  them, 
according  to  officials. 

•  Training:  As  of  February  2017,  VA  rolled  out  training  for  newly  hired 
attorneys  in  fiscal  year  2017,  which  includes  4  weeks  of  training  and  8 
additional  weeks  of  one-on-one  mentoring.  VA  also  stated  that  its 
Office  of  Knowledge  Management  was  expanded  with  additional  staff 
resources  to  ensure  training  materials  were  up  to  date.  However,  in 
November  2016,  officials  reported  that  the  Board  was  still  in  the 
process  of  updating  various  aspects  of  its  training  curriculum,  such  as 
how  to  support  conducting  work  in  a  virtual  environment,  which  is 
consistent  with  the  agency’s  plans  to  increase  telework  as  a  way  to 
manage  space  restrictions  for  new  staff.  In  its  comments  to  this  report, 
VA  did  not  provide  updates  on  this  effort,  as  of  February  2017. 

As  of  October  201 6,  it  was  unclear  how  the  Board’s  201 7  recruiting, 
hiring,  and  training  efforts  will  be  adjusted  to  support  the  agency’s 
proposed  hiring  surges  in  2018  or  its  proposed  process  reform.  For 
example,  the  Board  has  not  yet  determined  how  it  will  meet  the  space 
needs  for  any  additional  growth  associated  with  hiring  surges  proposed 
for  fiscal  year  2018,  although  more  detailed  planning  in  advance  might 
have  better  prepared  VA  for  bringing  aboard  242  FTEs  in  fiscal  year 
2017.  In  addition,  VA  officials  stated  in  February  2017  that  draft  training 
for  the  proposed  new  appeals  process  had  been  prepared  based  on 
statutory  language,  although  these  draft  documents  were  not  included  in 
VA’s  comments.  Federal  strategic  planning  guidance  calls  for  an  agency 


39 According  to  VA  officials,  one  practice  previously  identified  as  successful  was  using 
human  capital  experts  from  the  Veteran’s  Health  Administration  with  experience  in  large 
workforce  projects.  However,  the  Board  stated  it  was  not  using  these  experts  for  its  fiscal 
year  2017  hiring  surge. 

40 As  of  October  2016,  VA  hired  43  staff  and  planned  to  hire  25  new  staff  in  November  and 
December  2016,  accounting  for  holidays  when  hiring  efforts  will  slow,  and  about  50  new 
staff  in  the  first  few  months  of  201 7. 
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to  have  clear  plans  and  goals,  and  regularly  assess  its  human  capital 
approaches  through  assessments,  as  well  as  through  data-driven  human 
capital  management  to  improve  its  ability  to  maximize  the  value  of  human 
capital  investments  while  managing  related  risks.41  Conversely,  a  lack  of 
detailed  workforce  plans  and  mitigation  strategies  prior  to  proposed  hiring 
surges  in  2018,  as  well  as  potential  process  reform,  further  places  VA  at 
risk  of  not  being  ready  to  accommodate  another  quick  and  much  larger 
increase  in  staff,  or  to  train  them  in  accordance  with  either  the  legacy  or 
proposed  reform  process. 


VA  Collaborated  with 
Stakeholders  in 
Developing  Its  Proposed 
Streamlined  Appeals 
Framework,  but  Lacks 
Strategies,  Such  as  Pilot 
Testing,  to  Mitigate  Risks 


VA  collaborated  with  key  stakeholders  in  developing  its  proposed  appeals 
process  reform  framework  and  related  implementation  plans,  which  is 
consistent  with  sound  practices  for  business  process  redesign.42  Sound 
redesign  practices  suggest  coordinating  with  stakeholders  in  developing 
and  implementing  plans  to  obtain  and  maintain  buy-in  from  start  to  finish 
and  to  identify  and  address  disagreements.  In  developing  its  proposal, 
Board  and  VBA  officials  engaged  stakeholders  from  11  organizations — 
including  VSOs  that  represent  veterans  in  appeals  hearings  before  VBA 
and  the  Board — in  discussions  to  design  a  streamlined  appeals  process.43 
Officials  we  interviewed  from  three  of  four  VSOs,  all  of  whom  participated 
in  the  discussions,  noted  that  VA’s  resulting  process  proposal  addressed 
both  the  agency’s  desire  to  expedite  appeals  resolutions  and  stakeholder 
desires  that  the  new  process  be  fair  to  veterans.  For  example,  VA 
identified  and  prioritized  key  concerns  and  found  areas  of  consensus  with 
VSOs.  VA  officials  stated  that  they  plan  to  continue  to  discuss  appeals 
process  reform  (among  other  topics)  at  regular  meetings  with 
stakeholders,  during  which  they  will  have  an  opportunity  to  provide 
feedback  on  previously  unforeseen  issues.  VA  officials  said  that  as 
process  reform  is  implemented,  the  agency  will  invite  local  VSOs  to 


41  GAO,  Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and  Development 
Efforts  in  the  Federal  Government.  GAO-04-546G  (Washington  D.C.:  March  2004).  See 
also  GAO/OCG-00-14G. 

42See  GAO/AIMD-IO.1.15.  In  this  report,  we  refer  to  reengineering  as  appeals  process 
“redesign.” 

43The  organizations  were  the  American  Legion,  American  Veterans,  Disabled  American 
Veterans,  Military  Officers  Association  of  America,  National  Association  of  County  Veteran 
Service  Officers,  National  Association  of  State  Directors  of  Veterans  Affairs,  National 
Organization  of  Veteran  Advocates,  National  Veterans  Legal  Services  Program,  Paralyzed 
Veterans  of  America,  Veterans  of  Foreign  Wars,  and  Vietnam  Veterans  of  America. 
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training,  and  share  training  materials  and  provide  briefings  to  them  and 
other  stakeholder  groups. 

While  VA  has  achieved  broad  agreement  internally  and  with  VSOs  on  its 
proposed  process  reform,  there  are  several  unaddressed  gaps  in  VA’s 
business  case  for  implementation  that  introduce  the  risk  of  not  producing 
the  desired  results,  as  follows. 

•  To  develop  a  business  case  for  implementing  process  change,  sound 
redesign  practices  suggest  first  mapping  and  analyzing  the  target 
process  to  understand  the  cause  and  cost  of  performance 
breakdowns,  and  assessing  potential  barriers,  costs,  and  benefits  of 
alternative  processes.  This,  in  turn,  would  inform  the  selection  of  a 
feasible  alternative  with  a  high  return  on  investment,  and  the 
development  of  a  business  case  that  describes  benefits,  costs,  and 
risks.  However,  due  to  IT  limitations,  VA  lacks  data  to  inform  and 
confirm  its  understanding  of  the  root  causes  of  lengthy  time  frames. 
For  example,  VA  lacks  complete  historical  data  on  the  extent  to  which 
submission  of  new  evidence  and  multiple  decisions  and  appeals 
occur,  and  thus  cannot  determine  the  impact  of  its  current,  open- 
ended  process  on  appeals  decision  timeliness.44  To  shed  light  on  root 
causes,  VA  analyzed  10  appeals  decisions  that  it  found  took  a  long 
time  to  adjudicate  to  illustrate  extreme  examples  of  cases  being  re¬ 
reviewed  under  VA’s  open-ended  process — referred  to  by  VA  as 
“churning.”45  However,  VA  cannot  know  the  full  extent  to  which 
churning  might  be  occurring  because,  according  to  VA  officials,  the 
way  data  are  stored  made  it  difficult,  if  not  impossible,  to  assemble  a 
complete  historical  picture  prior  to  December  2012.  To  help  develop 
baseline  data,  VA  analyzed  the  average  number  of  decisions  per 
appeals  phase  for  several  recent  fiscal  years,  and,  according  to  VA 
officials,  they  are  still  endeavoring  to  piece  together  additional 
historical  baselines  for  performance. 


44VA  officials  noted  that  VACOLS  was  built  to  manage  workflow,  not  capture  data 
permanently.  Over  time,  as  data  fields  changed  or  were  repurposed,  prior  information  was 
lost  or  overwritten,  and  is  currently  unrecoverable.  Since  2012,  VA  has  taken  snapshots  of 
data,  and  thus  has  some  capabilities  to  analyze  more  recent  cases  involving  multiple 
appeals,  submission  of  new  evidence,  and  multiple  decisions  on  appeals. 

45According  to  VA  officials,  VA  undertook  this  analysis  in  response  to  a  congressional 
inquiry. 
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•  Further,  although  it  was  appropriate  for  VA  to  develop  its  proposed 
reform  in  consultation  with  internal  and  external  experts,  the  agency 
did  not  test  alternatives  using  data-driven,  cost-effective  methods 
suggested  by  sound  redesign  practices.46 

•  Finally,  as  noted  previously,  in  modeling  staff  resources  needed  under 
its  proposed  process  reform,  VA  relied  on  assumptions — about  the 
percent  of  veterans  who  will  refile,  will  appeal  to  the  Board,  and  will 
submit  new  evidence — that  have  direct  implications  for  projections  of 
appeals  workloads,  time  frames,  and  cost.  However,  VA  did  not 
perform  sufficient  sensitivity  analyses  to  help  estimate  a  range  of 
potential  outcomes — analyses  that  might  help  VA  understand  the 
likelihood  that  the  new  process  could  be  more  costly  and  time- 
consuming  in  practice  than  anticipated,  for  example,  if  a  higher 
percent  of  veterans  file  with  the  Board,  submit  new  evidence,  and 
request  hearings  than  expected. 

These  gaps  notwithstanding,  VA  made  some  progress  planning  for 
potential  implementation  of  proposed  process  reform  in  a  manner 
generally  consistent  with  sound  planning  practices  for  process  redesign 
and  change  management,  although  some  important  details  are  still 
absent.47  According  to  sound  planning  practices,  implementation  is  the 
most  difficult  phase  of  business  process  redesign.  An  agency  must 
manage  human  capital  and  technical  issues  as  it  turns  an  idea  into  reality 
and  overcomes  potential  resistance  to  change.  To  ensure  an  orderly 
transition,  sound  planning  practices  suggest  following  a  comprehensive 
implementation  plan  that  includes  several  key  activities,  such  as 
establishing  a  transition  team  and  developing  a  comprehensive  plan  to 
manage  implementation.48  Consistent  with  this,  as  of  October  2016,  the 


46Sound  redesign  practices  suggest  using  cost-effective  methods — such  as  prototyping, 
modeling  and/or  computer  simulation,  or  limited  pilot-testing — to  assess  the  costs, 
benefits,  and  feasibility  of  alternative  processes  that  will  inform  the  selection  of  a  feasible 
process  alternative  with  a  high  return  on  investment.  These  early  practices  would  also 
promote  buy-in,  and  help  determine  expected  performance  results. 

47ln  addition  to  GAO/AIMD-IO.1.15,  see  Insights  into  Organization  and  A  Guide  to  the 
Project  Management  Body  of  Knowledge  (PMBOK)  Guide. 

40 

According  to  sound  redesign  practices,  a  comprehensive  implementation  plan  should 
identify  tasks,  time  frames  and  needed  resources,  structure  the  roll-out  in  an  appropriate 
way,  assign  roles  and  responsibilities  for  implementation,  provide  a  means  for  collecting 
and  sharing  implementation  problems  and  solutions,  and  provide  for  close  monitoring 
during  implementation. 
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Board  and  VBA  had  identified  general  time  frames  and  offices 
responsible  for  key  implementation  efforts.  Based  on  its  staff  modeling 
efforts,  VA  also  identified  how  many  FTEs  it  expects  it  will  devote  to 
processing  cases  under  the  current  process  versus  a  new  one,  should  it 
be  implemented.  Also,  per  sound  planning  practices,  a  comprehensive 
plan  should  address  workforce  training  and  redeployment  issues 
(including  working  closely  with  employee  unions  to  minimize  potential 
adverse  effects).  Consistent  with  this,  as  of  October  2016,  VA  had 
outlined  general  steps  and  time  frames  for  training  of  staff  and 
communicating  with  the  unions. 

While  VA’s  high-level  implementation  plan  included  many  components 
suggested  by  sound  practices,  key  details  had  yet  to  be  addressed.  In 
particular,  VA’s  general  timetables  and  plans  to  date  have  not  addressed 
in  any  detail  how  it  will  implement  a  new  process  while  simultaneously 
working  to  reduce  the  appeals  inventory  under  the  current  process.49  For 
example,  the  agency  has  not  explained  how  or  who  will  track  timeliness  of 
appeals  of  the  old  compared  to  the  new  process,  and  how  decisions  will 
be  made  to  ensure  the  agency  is  devoting  an  appropriate  share  of 
resources  to  both  processes.  The  lack  of  a  detailed  plan  for  managing 
this  transition  exposes  the  agency  to  risk  that  veterans  whose  appeals 
are  pending  under  the  old  process  may  experience  significant  delays 
relative  to  those  under  a  new  process.  The  Board  recognized  the  need  to 
ensure  fairness  to  veterans  with  appeals  pending  under  the  current 
process,  and  indicated  that  while  legislation  is  pending  that  would 
authorize  a  new  process,  it  will  continue  to  develop  plans  for  managing 
the  two  processes  in  parallel. 

Sound  practices  for  process  redesign  and  change  management  also 
suggest  having  risk  mitigation  strategies — in  particular,  pilot  testing — to 
ensure  moving  successfully  to  full  implementation.50  Pilot  testing  provides 


49VA’s  plan  is  that  when  the  new  process  is  implemented,  pending  appeals  would 
continue  to  be  processed  under  the  “legacy”  or  current  process.  According  to  VA,  only 
appeals  of  decisions  where  the  notice  of  decision  was  provided  at  least  18  months 
following  enactment  of  appeals  reform  legislation  would  follow  the  new  appeals  process. 
As  of  December  2016,  VA  officials  said  they  were  also  considering  a  regulation  allowing 
veterans  whose  appeals  are  remanded  by  the  Board  under  the  legacy  process  to  opt  into 
the  new  framework. 

50See  GAO/AIMD-IO.1.15;  Insights  into  Organization,  and  Michael  L.  George,  David 
Rowlands,  Mark  Price,  and  John  Maxey;  and  The  Lean  Six  Sigma  Pocket  Toolbook:  A 
Quick  Reference  Guide  to  Nearly  1 00  Tools  for  Improving  Process  Quality,  Speed,  and 
Complexity.  (New  York:  McGraw-Hill,  2005). 
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agencies  opportunities  to  evaluate  the  soundness  of  new  processes  in 
actual  practice  on  a  smaller  scale,  and  to  refine  performance  measures, 
collect  and  share  implementation  problems  and  solutions,  correct 
problems  with  the  new  design  and  refine  the  process  prior  to  full 
implementation,  and  build  capacity  among  unit  managers  to  lead  change. 
Sound  redesign  and  change  management  practices  both  suggest  that 
pilot  tests  should  be  rigorously  monitored  and  evaluated,  and  that  further 
roll-out  occur  only  after  the  agency’s  transition  team  has  taken  any 
needed  corrective  action  and  determined  that  the  new  process  is 
achieving  previously  identified  success  criteria.51  As  noted  above,  pilot 
testing  is  not  the  only  method  of  achieving  these  risk  mitigation  goals,  but 
sound  planning  practices  suggest  pilot  testing  is  an  important,  often 
necessary  approach  for  ensuring  successful  implementation  when 
undertaking  significant  institutional  change.52 

Contrary  to  sound  practices,  VA  officials  stated  they  do  not  want  to  pilot 
proposed  appeals  reform,  even  though  VA’s  proposed  reform  can  be 
considered  complex.  VA’s  reform  plans  qualify  as  complex  because  in 
addition  to  implementing  a  new  process,  the  agency  must  still  manage  a 
large  inventory  of  appeals  under  the  old  process  while  hiring  and  training 
a  large  number  of  staff  and  implementing  IT  improvements.53  Occurring 


51Sound  change  management  practices  suggest  that  after  learning  from  pilot  testing, 
organizations  need  to  decide  how  to  scale  up  any  changes.  Factors  to  consider  in 
deciding  how  to  scale  up  in  connection  with  risk  mitigation  strategies  include  whether:  the 
organization  faces  a  crisis,  there  is  resistance  to  change,  the  resources  are  available  to 
implement  change,  and  implementation  requires  customization.  See  Insights  into 
Organization. 

52 As  discussed  previously,  prior  to  implementation,  agencies  can  mitigate  risks  associated 
with  process  change  by  first  assessing  potential  barriers,  costs,  and  benefits  of  alternative 
processes  to  help  ensure  selection  of  a  feasible  process  with  a  high  return  on  investment. 
These  early  practices  would  also  promote  buy-in,  and  help  determine  expected 
performance  results.  After  selecting  the  most  promising  concept,  pilot  testing  prior  to  full 
implementation  provides  an  opportunity  for  testing  and  refining  the  new  process  in  actual 
practice,  and  building  support  for  full  implementation.  We  recognize  that  some  risk 
mitigation  strategies  would  be  within  VA’s  current  authority  while  others — such  as 
piloting — would  require  explicit  congressional  authorization. 

53As  noted  previously,  the  Board  has  already  encountered  challenges  finding  space  to 
accommodate  new  staff  being  hired  in  fiscal  year  2017,  despite  considering  lessons 
learned  from  a  prior  hiring  surge  within  the  agency.  Further,  we  recently  reported  that  VBA 
encountered  complexities — including  competing  workload  priorities  and  scope 
expansion — that  resulted  in  decisions  that  delayed  implementation  of  a  new  claims 
system.  See  GAO,  Veterans  Benefits  Management  System:  Ongoing  Development  and 
Implementation  Can  Be  Improved;  Goals  are  Needed  to  Promote  Increased  User 
Satisfaction.  GAO-15-582  (Washington,  D.C.:  September  1, 2015). 
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together,  these  efforts  involve  significant  change  and  uncertainty  and  will 
require  management  oversight  across  a  broad  range  of  efforts.  In 
addition,  VA’s  proposed  process  reform  and  other  initiatives  affect  VBA’s 
regional  offices  spread  across  the  country,  as  well  as  the  centrally  located 
Board,  thereby  further  increasing  complexity  of  implementation.  VA 
officials  also  stated  that  the  proposed  process  reform,  which  has  been 
thoroughly  vetted  with  stakeholders,  has  broad  support,  and  noted  their 
view  that  the  risk  of  fully  implementing  change  is  outweighed  by  the  cost 
of  delay.  VA’s  rationale  for  not  pilot  testing  centers  on  what  they  describe 
as  widespread  consensus  that  the  current  process  is  “fundamentally 
broken”  and  provides  “inadequate  service  to  veterans  with  a  high 
percentage  of  wasted  effort.”  VA  assumed  that  a  pilot  test  authorized  by 
Congress  would  include  a  sunset  date  with  a  default  reversion  to  the 
current  system,  which  they  said  would  introduce  uncertainty  into  the 
agency’s  planning  efforts  and  a  reliance  on  subsequent,  time  consuming 
legislation  before  the  conclusion  of  the  pilot.  VA  officials  stated  that 
piloting  with  a  sunset  date  would  require  the  agency  to  expend  additional 
resources  and  time  to  conduct  parallel  planning  for  reverting  to  the  old 
system  upon  the  sunset  date.  VA  stated  that  pilot  testing  the  new  process 
for  some  veterans  would  be  perceived  as  inequitable,  despite  VA  having 
previously  supported  pilot  testing  a  new  appeals  process.54  VA  officials 
concluded  that  they  have  not  identified  any  risk  that  would  justify  a  pilot, 
and  indicated  that  they  plan  to  mitigate  risk  with  a  strong  implementation 
plan. 

While  VA  has  made  a  compelling  case  for  reforming  the  appeals  process, 
as  noted  previously,  VA’s  business  case  for  its  proposed  reform  in  some 
instances  relies  on  unproven  assumptions  and  limited  analyses  of  its 
current  process  that  introduces  risk  in  VA’s  plans  for  full  implementation. 
Importantly,  VA  assumes  that  because  the  current  framework  is 
“fundamentally  broken,”  its  proposed  new  framework  will  necessarily  be  a 
better  option.  However,  VA  made  this  decision  lacking  complete  data  on 
the  root  cause  of  lengthy  appeals  under  the  current  process,  and  without 
analyzing  barriers,  costs  and  benefits  of  feasible  alternatives  using  cost 


54VA  previously  supported  an  opt-in  pilot  program  that  would  have  tested  a  new  appeals 
process,  in  the  form  of  fully  developed  appeals  pilot  legislation  that  was  previously 
introduced  in  Congress.  See,  e.g.,  H.R.  800,  114th  Cong.  (2015).  This  legislation  would 
have  given  veterans  the  option  to  file  fully  developed  appeals.  Under  this  option,  veterans 
would  have  been  limited  in  the  new  evidence  they  could  present,  and  the  Board,  to  the 
extent  practicable,  would  have  decided  such  cases  within  1  year  of  the  appeal  being  filed. 
The  bill  was  not  enacted  and  VA  withdrew  its  support  for  this  pilot  in  favor  of  its  current 
proposal  to  streamline  the  appeals  process. 
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effective  methods,  such  as  computer  simulations.  VA  correctly  notes  that 
pilot  testing  prior  to  full  implementation  would  slow  down  an  overhaul  of 
the  current  system,  thus  countering  the  short-term  net  benefit  that  the 
agency  expects  to  realize  from  such  an  overhaul.  However,  VA  has  not 
acknowledged  that  pilot  testing  the  new  process  in  a  more  limited  fashion 
could  greatly  increase  the  probability  of  long-term  success  by  decreasing 
the  chance  that  a  new  system  will  experience  unanticipated  problems  that 
are  potentially  more  widespread  and  therefore  costly  to  remedy.  The 
inclusion  of  risk  mitigation  strategies  such  as  pilot  testing  does  not,  as  VA 
asserts,  “imply  that  the  status  quo  is  not  in  dire  need  of  sweeping  reform” 
but  rather  balances  the  urgency  of  the  current  problem,  the  technical 
complexity  of  an  overhaul,  and  the  potential  for  unforeseeable 
complications. 

In  light  of  this  and  the  previously  discussed  inconsistencies  in  following 
sound  planning  practices,  pushing  forward  with  full  implementation 
without  testing  how  process  reform  unfolds  and  interacts  with  other  efforts 
in  actuality,  may  lead  VA  to  experience  implementation  challenges  and 
setbacks  that  could  undermine  efficiencies  and  other  outcomes  resulting 
from  planned  reforms.  In  contrast,  if  VA  were  to  pilot  test  the  proposed 
appeals  process  reform,  implementation  problems  encountered  could  be 
identified  and  resolved  prior  to  full  implementation.  This  could  lead  to 
smoother  implementation  and  better  outcomes  overall.  Further,  resources 
that  would  otherwise  be  diverted  to  full  implementation  of  process  reform 
across  the  organization  could  be  focused  on  its  current  inventory  of 
appeals.  VA  will  also  have  additional  time  and  managerial  capacity  to 
recruit  and  train  new  staff  and  develop  and  implement  a  communication 
and  outreach  strategy  in  time  for  full  implementation  of  the  new  process. 
Finally,  if  risk  mitigation  strategies  demonstrated  that  process  reform 
would  be  more  costly  and  detrimental  to  time  frames  and  workloads  than 
predicted,  a  decision  to  modify  or  fix  the  process  at  that  juncture  would  be 
made  with  more  information  and  less  impact  on  the  agency  overall. 

Whether  VA  conducts  pilot  testing  or  not,  VA  has  not  yet  developed  a 
plan  for  closely  monitoring  implementation  or  developed  a  strategy  for 
assessing  the  success  of  its  proposed  process  reform.  Sound  planning 
and  redesign  practices  suggest  that  the  transition  team  develop  metrics 
and  data  gathering  procedures,  define  success  criteria,  measure 
performance  carefully,  and  take  corrective  action  of  any  pilot  test  before 
proceeding  to  full  implementation.  Sound  practices  also  suggest  the 
agency  develop  meaningful  performance  measures — generally  a  mix  of 
outcome,  output,  and  efficiency  measures — tied  to  overall  goals  of  the 
project,  and  that  project  goals  include  a  mix  of  intermediate  goals  to  be 
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met  at  various  stages  during  the  implementation  phase.  That  way,  the 
agency  can  start  to  show  a  return  on  investment  in  the  early  stages  of 
implementation.55  To  date,  VA  has  identified  several  broad  metrics 
generally  reflecting  outcomes,  output,  and  efficiency — such  as  veteran 
survey  results,  wait  times,  and  inventories — that  it  plans  to  use  to  track 
and  assess  process  improvements.  VA  also  established  separate 
timeliness  measures  for  the  Board  and  VBA  that  it  will  use  in  its  annual 
performance  reports.  While  these  broad  metrics  and  goals  are 
appropriate,  they  fall  short  of  sound  practices  for  monitoring  and 
assessing  process  change  in  several  respects. 

First,  VA  has  not  developed  a  dashboard  or  balanced  scorecard,  or 
otherwise  identified  how  it  will  closely  monitor  progress,  evaluate  its 
efficiency  and  effectiveness,  and  identify  trouble  spots.  For  example, 
although  VA  has  stated  that  it  is  developing  a  dashboard  to  measure 
performance  under  its  proposed  appeals  process,  VA  has  not  yet 
indicated  whether,  how,  and  with  what  frequency  it  will  monitor  wait  times 
and  inventories  under  the  new  versus  current  processes.  As  a  result,  it  is 
not  clear  how  VA  will  determine  whether  veterans  with  appeals  pending 
under  the  current  process  are  receiving  equitable  treatment  and  not 
experiencing  significant  delays  relative  to  those  under  the  new  process.  It 
is  also  unclear  the  extent  to  which  VA  will  systematically  monitor  staff 
productivity  and  IT  processing,  which  may  affect  its  ability  to  determine 
whether  assumptions  are  being  met  to  help  pinpoint  corrective  action 
(e.g.,  whether  staff  need  more  training,  VA’s  communication  and  outreach 
efforts  are  working  as  expected,  or  process  reform  itself  is  achieving 
desired  results).  Further,  VA  has  not  established  interim  goals  or  criteria 
for  success  to  help  determine  whether  initial  implementation  is  achieving 
intended  results.56  Interim  goals  and  criteria  could  include  specific 
timeliness  improvements  for  process  steps  and  outcomes,  such  as 
average  time  for  VBA  or  the  Board  to  reach  decisions  under  new  appeals 
options.  If  VA  pursues  pilot  testing,  such  goals  or  success  criteria  will  help 


55For  example,  sound  change  management  practices  suggest  tracking  near-term 
business  performance,  and  using  an  integrated  set  of  metrics  that  span  operational 
improvements,  health  indicators  (e.g.  staff  engagement),  and  bottom-line  results.  Metrics 
could  include  cycle  time,  quality,  pulse  checks  of  staff,  and  other  business  outcomes  to 
ensure  unforeseen  consequences  are  not  occurring  elsewhere  in  the  organization. 

56VA  noted  that  as  of  February  201 7  it  had  established  timeliness  goals  for  three  of  the 
four  options.  Because  these  goals  represent  average  processing  times,  they  are  less 
useful  for  flagging  early  progress,  percent  of  appeals  meeting  the  goal,  and  trouble  spots. 
Further,  the  agency  did  not  set  a  goal  for  appeals  to  the  Board  involving  new  evidence 
and  hearings — an  option  that  will  likely  require  more  processing  time  than  the  others. 
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determine  whether  the  new  process  is  sufficiently  successful  to  justify  full 
implementation. 

Second,  although  streamlining  the  current  open-ended  process  was 
central  to  VA’s  business  case  for  its  proposed  process  reform,  as  noted 
previously,  VA  currently  lacks  sufficient  data  to  assess  the  extent  to  which 
process  reform  will  improve  on  the  open-ended  nature  of  the  current 
process.  VA  officials  said  that  they  plan  to  work  to  incorporate  capabilities 
into  Caseflow  to  piece  together  historical  baselines  for  performance.  VA 
also  plans  to  develop  new  baseline  and  historical  data  on  aspects  of  the 
appeals  process  that  affect  the  timeliness  of  final  decisions  so  that  they 
can  be  compared  to  the  new  process.  While  these  are  positive  steps,  it 
remains  to  be  determined  how  or  whether  VA  will  be  able  to  measure  the 
extent  to  which  its  proposed  process — which  would  allow  the  veteran  to 
appeal  multiple  times — is  an  improvement  over  “churning”  associated  with 
the  old  process. 

Lastly,  the  new  timeliness  measures  that  VA  plans  to  report  to  Congress 
and  the  public  lack  transparency  on  whether  overall  appeals  resolution 
timeliness  is  improving  from  the  veterans’  perspective.  In  its  fiscal  year 
2015  performance  report,  VA  stopped  reporting  its  average  appeals 
resolution  time  measure,  which  included  appeals  decisions  made  by  both 
VBA  and  the  Board.  VA  officials  said  they  considered  this  measure 
inadequate  because  neither  VBA  nor  the  Board  has  full  control  over 
making  improvements  to  performance  under  this  measure.  VA  officials 
told  us  the  measure  does  not  appropriately  provide  insight  into  the 
appeals  process  because  of  the  variety  of  appeals  paths  and  wait  times 
veterans  experience.  However,  the  combined  measure  would  provide  a 
basis  for  comparing  timeliness  under  the  old  versus  new  process,  and 
would  provide  historical  perspectives  on  changes  in  timeliness  from  the 
point  of  view  of  a  veteran  who  may  file  appeals  with  both  VBA  and  the 
Board  before  his  or  her  case  is  resolved.  VA  officials  stated  the  agency 
will  continue  to  track  this  measure  internally,  but  they  will  not  include  it  in 
VA’s  annual  performance  reports.  Instead,  they  plan  to  report  on  VBA  and 
Board  timeliness  separately.  VA  also  stated  that  it  will  not  use  this 
measure  to  evaluate  success  of  the  new  process  because  it  considers  a 
timeliness  measure  covering  both  VBA  and  the  Board  to  be  inappropriate. 
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VA  Has  Taken  Steps 
Consistent  With  Sound 
Planning  Practices  while 
Updating  Its  IT,  but  Lacks 
Clear  Schedules  for 
Implementing  IT  Updates 

•  Setting  goals  and  objectives:  VA  plans  clearly  lay  out  the  need  for 
replacing  VACOLS  and  set  forth  how  Caseflow  will  address  the 
shortcomings  of  VA’s  current  IT  system.  Its  plans  also  lay  out  a  set  of 
broad  milestones  in  terms  of  the  capabilities  that  will  be  added  to 
Caseflow  in  the  future  and  the  ultimate  retirement  of  VACOLS. 

•  Identifying  and  mitigating  potential  risks:  VA  planning  documents 
identify  a  number  of  risks  (such  as  staffing  shortfalls  and  technical 
delays)  and  strategies  to  mitigate  them.  In  addition,  VA  is  developing 
Caseflow  in  an  agile  process,  which  officials  say  will  allow  VA  to 
continually  add  new  capabilities  and  be  responsive  to  changing 
agency  needs.  In  addition,  VA  officials  told  us  that  rather  than  replace 
VACOLS  at  once,  the  various  functions  in  VACOLS  will  be 
reproduced  and  tested  in  Caseflow  iteratively,  and  each 
corresponding  function  in  VACOLS  will  be  left  intact  until  there  is 
reasonable  assurance  that  there  will  be  no  impact  to  VA. 

•  Measuring  performance:  VA  plans  to  develop  metrics  for  each  new 
component  of  Caseflow  that  is  implemented.  For  instance,  VA  has 
developed  metrics  for  the  two  components  that  were  developed  in 
fiscal  year  2016 — electronic  transfer  of  cases  to  the  Board  and  a 
system  to  electronically  access  documents  from  VBMS — which 
specifically  assess  the  performance  and  effect  of  those  components. 
As  mentioned  earlier,  VA  also  plans  to  create  a  Caseflow  dashboard 
that  will  provide  metrics  on  the  effect  of  IT  improvements  on 
timeliness  of  the  appeals  process. 

•  Identifying  organizational  roles  and  responsibilities:  VA  entered 
into  a  memorandum  of  understanding  with  DSVA  and  the  Board  that 
outlines  priorities,  and  a  working  relationship  for  developing  Caseflow. 
In  addition,  the  memorandum  states  that  DSVA  requires  all  initiative 


VA  has  generally  planned  the  implementation  of  its  Caseflow  appeals 
system  consistent  with  sound  planning  practices.  Working  with  U.S. 
Digital  Service  at  VA  (DSVA)57 — the  group  tasked  with  developing 
Caseflow — VA  outlined  an  approach  that  has  a  clear  scope  and  purpose, 
which  is  to  better  process  appeals  in  a  paperless  environment  and 
improve  automation  and  productivity.  The  actions  consistent  with  sound 
IT  planning  practices  include:58 


57DSVA  is  part  of  VA’s  Office  of  Information  and  Technology. 
58See  GAO-04-394G. 
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partners  within  VA  to  have  a  single  point  of  contact  with  the  authority 
to  make  decisions  on  behalf  of  their  component. 

While  VA’s  plans  for  replacing  VACOLS  take  steps  to  mitigate  risks,  they 
currently  do  not  include  consideration  of  the  timing  and  implications  of 
VA’s  proposed  reform  efforts.  Federal  internal  control  standards  state  that 
program  managers,  in  seeking  to  achieve  program  objectives,  should 
define  objectives  clearly  to  enable  the  identification  of  risks.  This  includes 
clearly  defining  what  is  to  be  achieved  and  the  time  frames  for 
achievement.  Additionally,  IT  investment  best  practices  stress  the  need 
for  oversight  regarding  a  project’s  progress  towards  predefined  schedule 
expectations.  This  oversight  also  includes  systems  to  make  corrections 
regarding  schedule  and  performance  slippages.  Although  VA  has  laid  out 
the  broad  capabilities  it  would  like  to  incorporate  into  Caseflow  going 
forward,  VA  has  not  developed  a  schedule  for  completing  Caseflow. 
Specifically,  VA  could  not  provide  us  with  firm  time  frames  for  when 
different  capabilities  will  be  active  in  Caseflow.  As  the  Caseflow  effort 
lacks  time  frames,  VA  cannot  ensure  that  the  system  will  be  completed  in 
time  to  support  the  implementation  of  proposed  reforms. 

Further,  VA’s  lack  of  time  frames  for  developing  Caseflow  may  increase 
the  risk  of  additional  costs  if  the  system  cannot  be  developed  as  quickly 
as  anticipated.  Sound  practices  specific  to  project  scheduling  state  that 
project  planning  is  the  basis  for  controlling  and  managing  project 
performance,  including  managing  the  relationship  between  cost  and 
time.59  In  a  prior  GAO  report  on  VBMS  development — which  was  also 
developed  in  an  agile  process — we  reported  that  the  agency  encountered 
some  delays  with  its  initial  deployment  of  key  functions  of  VBMS,  and  that 
its  lack  of  a  schedule  made  it  difficult  to  hold  program  managers 
accountable  for  meeting  time  frames  and  demonstrating  progress.60 

In  addition,  VA  has  not  started  planning  and  determining  the  changes  that 
would  be  needed  for  Caseflow  if  and  when  appeals  reforms  are 
implemented.  VA  staff  said  that  the  agile  approach  they  are  using  allows 
them  to  quickly  respond  to  changing  needs,  and  VA  Office  of  Information 


59GAO,  Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules.  GAO-1 6-89G 
(Washington,  D.C:  December  201 5). 

60We  reported  that  some  delays  in  completing  functions  of  VBMS  were  due  to  factors 
such  as  decisions  to  enlarge  VMBS’  scope  over  time  and  to  delay  plans  to  establish  an 
electronic  work  queue,  so  that  VBA  could  fully  focus  on  eliminating  its  claims  backlog 
within  established  goals.  See  GAO-15-582. 
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and  Technology  officials  told  us  that  they  will  not  begin  planning  for  such 
changes  until  reform  legislation  is  passed.  As  stated  earlier,  sound  IT 
planning  practices  suggest  that  implementation  plans  include  specific 
time  frames  and  approaches  needed  to  implement  new  systems,  as  well 
as  the  consideration  of  potential  risks  and  mitigation  strategies.  As  such, 
and  given  the  absence  of  a  schedule  for  completing  Caseflow,  VA  further 
risks  having  an  IT  system  that  is  not  completed  in  a  timely  manner  or, 
even  if  in  place  in  time,  falls  short  of  meeting  VA’s  needs. 


Conclusions 


With  an  already  large  inventory  of  pending  appeals — and  expectations  of 
further  growth — VA  has  taken  steps  to  bolster  capacity  and  improve  the 
efficiency  and  effectiveness  of  its  disability  appeals  process.  Specifically, 
VA  hired  and  proposed  hiring  more  staff,  is  moving  forward  with  plans  to 
upgrade  its  IT  systems,  and  has  proposed  bold  reforms  to  streamline  its 
appeals  process.  In  planning  and  executing  these  approaches,  VA  took 
several  positive  steps  in  line  with  sound  planning  practices — such  as 
comparing  different  options  for  increasing  future  staffing  resources, 
collaborating  with  external  stakeholders  to  develop  a  streamlined  process 
proposal,  and  outlining  a  vision  for  upgrading  outdated  IT  systems. 

Nonetheless,  VA’s  plans  do  not  account  for  the  significant  challenges  that 
remain.  Above  all,  its  proposal  to  implement  appeals  reform  at  the  Board 
and  across  VBA’s  regional  offices  is  ambitious,  and  as  a  result,  VA  may 
be  exposing  itself  to  unforeseen  risks  and  setbacks  that  could  slow 
progress  toward  improving  appeals  decision  timeliness.  More  specifically, 
VA  has  proposed  implementing  process  reform  while  also  hiring  more 
staff  and  upgrading  its  IT,  which  are  challenging  efforts  in  their  own  right. 
Additionally,  VA  does  not  have  any  plans  to  pilot  test  its  proposal — a 
sound  and  often  necessary  practice  for  experiencing,  evaluating,  and 
refining  significant  institutional  change  on  a  smaller  scale  prior  to  full 
implementation.  At  the  same  time,  VA  plans  for  hiring  more  staff  and 
upgrading  IT  lack  key  details  (for  example,  on  how  VA  will  train  and  find 
working  space  for  new  staff,  or  a  schedule  for  when  and  how  system 
changes  might  be  integrated  with  the  proposed  streamlined  process), 
exposing  VA  to  risks  of  delays,  inefficiencies,  or  other  setbacks  caused  by 
not  anticipating  needs  or  a  misalignment  of  efforts.  VA  also  did  not 
sufficiently  apply  sensitivity  analysis  when  projecting  staffing  needs  with 
or  without  process  reform,  which  could  affect  the  agency’s  ability  to 
mitigate  any  potential  risks  if  assumptions  are  not  met.  Lastly,  VA  lacks  a 
robust  monitoring  plan  to  help  assure  that  unforeseen  problems  will  be 
quickly  and  effectively  addressed,  and  has  not  yet  developed  a  strategy 
with  appropriate  interim  goals  for  process  reform,  and  overall  goals  for 
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appeals  process  timeliness,  to  gauge  whether  the  agency’s  efforts  are 
having  the  desired  result  and  reflect  an  improvement  over  prior  practices. 
Until  VA  incorporates  these  sound  planning  practices,  the  agency  lacks 
reasonable  assurance  that  its  proposed  reform  will  improve  the  overall 
efficiency  of  the  appeals  process  and  timeliness  of  disability  appeals 
decisions. 


Matter  for 

Congressional 

Consideration 


To  improve  VA’s  ability  to  successfully  implement  appeals  process 
reform,  Congress  should  consider  requiring  that  reforms  of  the  VA 
disability  appeals  process  be  subject  to  a  pilot  test.  To  aid  in  the 
development  of  such  a  pilot  test,  Congress  could  require  the  Secretary  of 
Veterans  Affairs  to  propose  options  that  would  allow  the  agency  the 
flexibility  to  test  and  refine  the  new  process  in  a  cost-effective  and 
efficient  manner,  while  ensuring  pre-established  interim  goals  and 
success  criteria  are  being  met  prior  to  full  implementation. 


Recommendations  for  ^urt*ier  a''9n  eff°rts  to  address  appeals  workload  and  improve 

timeliness  of  decisions,  and  reduce  the  risk  that  efforts  will  not  go  as 
Executive  Action  planned,  we  recommend  that  the  Secretary  of  Veterans  Affairs  direct  the 

Under  Secretary  for  Benefits;  the  Chairman,  Board  of  Veterans’  Appeals; 
and  the  Chief  Information  Officer,  as  appropriate,  to: 

1 .  Ensure  development  of  a  timely,  detailed  workforce  plan  for  recruiting, 
hiring  and  training  new  hires.  In  particular,  this  plan  should:  (1)  include 
detailed  steps  and  timetables  for  updating  training  curriculum  (such  as 
preparing  decisions  in  a  virtual  environment)  and  ensuring  office 
space  (such  as  telework  guidance);  and  (2)  incorporate  risk  mitigation 
strategies  that  consider  how  the  timing  of  recruitment  and  training 
dovetails  with  uncertain  time  frames  for  implementing  a  new  appeals 
process. 

2.  Develop  a  schedule  for  IT  updates  that  explicitly  addresses  when  and 
how  any  process  reform  will  be  integrated  into  new  systems  and  when 
Caseflow  will  be  ready  to  support  a  potential  streamlined  appeals 
process  at  its  onset. 

3.  Conduct  additional  sensitivity  analyses  based  on  the  assumptions 
used  in  projection  models  to  more  accurately  estimate  future  appeals 
inventories  and  timeliness.  In  doing  so,  consider  running  additional 
analyses  on  how  these  factors,  in  conjunction  with  one  another,  may 
affect  the  timeliness  and  cost  of  deciding  pending  appeals. 
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4.  Develop  a  more  robust  plan  for  closely  monitoring  implementation  of 
process  reform  that  includes  metrics  and  interim  goals  to  help  track 
progress,  evaluate  efficiency  and  effectiveness,  and  identify  trouble 
spots. 

To  better  understand  whether  appeals  process  reform,  in  conjunction  with 
other  efforts,  has  improved  timeliness,  we  recommend  the  Secretary  of 
Veterans  Affairs  direct  the  Under  Secretary  for  Benefits;  the  Chairman, 
Board  of  Veterans’  Appeals;  and  the  Chief  Information  Officer,  as 
appropriate  to: 

5.  Develop  a  strategy  for  assessing  process  reform — relative  to  the 
current  process — that  ensures  transparency  in  reporting  to  Congress 
and  the  public  on  the  extent  to  which  VA  is  improving  veterans’ 
experiences  with  its  disability  appeals  process. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  the  report  to  VA  for  its  review.  In  written 
comments,  VA  disagreed  with  one  of  our  recommendations  and  agreed  in 
principle  with  the  other  five.  We  have  reproduced  VA’s  comments  in 
appendix  I  and  have  incorporated  them — as  well  as  technical  comments 
provided  by  VA — into  the  report,  as  appropriate. 

In  its  comments,  VA  agreed  with  us  that  improving  the  efficiency  and 
effectiveness  of  its  appeals  process  is  an  ambitious  undertaking,  and  we 
commend  VA  for  the  many  steps  it  has  taken,  including  collaborating  with 
stakeholders  to  develop  the  framework  for  a  new  process.  We  agree  that 
obtaining  the  consensus  of  internal  and  external  experts — including 
veterans  service  organizations — demonstrates  important  progress.  We 
disagree,  however,  that  such  consensus  negates  the  need  for  more 
detailed  plans  and  robust  risk  mitigation  strategies.  While  it  is  true  that  VA 
has  made  noteworthy  progress  developing  an  implementation  plan  to 
guide  its  efforts,  we  found  the  plan  lacked  important  details,  such  as:  how 
VA  will  monitor  for  interim  success  and  trouble  spots,  including  whether 
the  agency  has  appropriately  distributed  resources  among  the  new  and 
old  processes;  how  it  will  mitigate  risk  of  implementation  challenges  or 
setbacks,  and  reduce  their  negative  impact;  and  how  it  will  measure 
whether  the  new  process  is  improving  overall  appeals  resolution 
timeliness  from  the  veteran’s  perspective. 

VA  officials  also  said  that  VA  has  extensive  experience  in  organizational 
change  management,  but  it  is  not  clear  how  some  of  the  practices  VA 
used  in  past  transformation  efforts  are  applicable  to  appeals  reform,  and 
we  are  concerned  that  VA  could  not  provide  further  information  on  what 
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these  practices  include  or  how  they  are  relevant.  We  believe 
implementing  all  of  our  recommendations  will  increase  the  likelihood  that 
VA’s  efforts  to  improve  the  efficiency  and  effectiveness  of  its  appeals 
process  will  be  successful.  For  the  five  recommendations  that  VA 
concurred  with  in  principle,  VA  described  planned  actions  to  address 
them  and  stated  that  it  also  considered  the  actions  complete  and 
requested  we  close  the  recommendations.  However,  we  believe  VA  still 
needs  to  take  actions  to  address  those  recommendations,  as  noted  more 
fully  below. 

VA  disagreed  with  a  draft  recommendation  that  it  incorporate  pilot  testing 
of  its  proposed  appeals  process  into  implementation  plans  and  pursue 
necessary  legislative  authority.  In  its  comments,  VA  noted  that  the 
appeals  process  is  broken  and  that  piloting  a  new  process  would  result  in 
further  delays  to  veterans  appealing  their  disability  decisions.  VA 
disagreed  with  GAO’s  finding  that  it  had  proposed  the  new  process 
without  analyzing  feasible  alternatives,  noting  that  the  agency  designed 
the  new  process  based  on  the  collective  experience  of  internal  and 
external  experts,  and  that  these  experts  reached  consensus  on  a  new 
design  that  will  be  beneficial  to  veterans,  the  agency,  and  taxpayers, 
among  others.  VA  noted  that  it  has  carefully  assessed  risks,  identified  a 
number  of  risk  mitigation  strategies,  modeled  a  number  of  different 
scenarios,  and  developed  a  detailed  implementation  plan. 

When  we  reviewed  these  efforts,  however,  we  found  three  primary 
shortcomings  that  need  to  be  addressed.  First,  VA  did  not  have  the  data  it 
needs  to  fully  understand  the  extent  to  which  the  current  process  has 
contributed  to  lengthy  appeals  time  frames,  which  raises  questions  about 
whether  the  proposed  process  will  address  the  root  cause  or  causes  of 
untimely  appeals  decisions.  Specifically,  VA  lacks  historical  data  on  the 
extent  to  which  the  introduction  of  new  evidence  increases  time  frames. 
Second,  VA’s  list  of  potential  risks  and  risk  mitigation  strategies  did  not 
always  include  steps  for  mitigating  the  identified  risks.61  Third,  we  found 
that  VA’s  implementation  plans  lacked  details  on  how  it  will  carry  out  key 
aspects  of  appeals  reform,  including  how  it  will  monitor  the  timeliness  of 
appeals  decisions  under  the  old  process  compared  to  the  new  appeals 


61  In  particular,  regarding  the  risks  that  veteran  behavior  under  the  new  Framework  is 
unknown  and  may  vary  from  VA’s  assumptions.  For  example,  VA’s  assumptions  about 
veterans’  likelihood  of  appealing  directly  to  the  Board  might  be  too  high  or  too  low.  VA’s 
stated  risk  mitigation  strategy  was  to  use  conservative  assumptions  developed  by  agency 
and  stakeholder  experts,  and  to  monitor  the  actual  experience  and  update  models 
accordingly.  VA  did  not  identify  steps  it  might  take  if  more  veterans  request  a  higher  level 
or  more  reviews  than  assumed. 
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process,  while  also  hiring  additional  staff  and  integrating  changes  into  the 
Caseflow  IT  system  as  discussed  below.  In  VA’s  comments,  it  also  stated 
that  piloting  a  new  appeals  process  “would  raise  constitutional  issues  and 
prompt  litigation.”  We  acknowledge  that  changing  an  adjudicatory  process 
for  determinations  of  benefits  may  prompt  litigation.  However,  VA  has  not 
clearly  articulated  why  pilot  testing  as  a  category  is  unconstitutional  or 
why  pilot  testing  poses  unique  constitutional  issues.  Further,  as  noted  in 
the  report  and  in  VA’s  comments,  VA  previously  supported  H.R.  800  in 
the  1 14th  Congress,  which  would  have  directed  VA  to  conduct  an  opt-in 
pilot  process  where  a  veteran  could  present  a  limited  amount  of  new 
evidence  and  the  Board,  to  the  extent  practical,  would  decide  cases 
within  one  year.  While  GAO  did  not  take  a  position  on  that  bill,  or  its 
specific  approach  to  pilot  testing,  changes  of  this  magnitude  in  such  a 
complex  program  justify  some  form  of  pilot  testing  to  ensure  process 
reform  is  implemented  successfully  and  ultimately  achieves  VA’s  goals. 

As  noted  in  the  report,  pilot  testing  is  recognized  to  be  a  sound  planning 
practice  and  an  important,  often  necessary  approach  for  ensuring 
successful  implementation  when  undertaking  significant  institutional 
changes. 

Until  VA  pilot  tests  its  appeals  reforms,  it  will  lack  data  to  properly  plan  for 
and  overcome  the  challenges  that  will  likely  arise  during  implementation. 
For  example,  VA  may  encounter  difficulties  making  needed  process 
changes  while  simultaneously  implementing  other  logistical  requirements, 
such  as  hiring  and  training  new  staff  and  updating  its  IT  system.  By  not 
pilot  testing,  VA  is  missing  a  valuable  opportunity  to  refine  its 
implementation  strategy  by  first  seeing  how  process  reform  will  unfold  on 
a  smaller  scale.  We  believe  that  the  potentially  negative  consequences  of 
delaying  full  implementation  are  far  outweighed  by  the  benefits  that  can 
be  realized  through  piloting.  For  example,  piloting  could  help  avoid  delays 
and  expenses  caused  by  the  need  to  re-work  the  process  after  full  scale 
implementation.  In  light  of  VA’s  disagreement  with  our  draft 
recommendation,  we  removed  the  recommendation  and  now  pose  a 
matter  for  congressional  consideration.  Specifically,  to  improve  VA’s 
ability  to  successfully  implement  appeals  process  reform,  Congress 
should  consider  requiring  that  reforms  of  the  VA  disability  appeals 
process  be  subject  to  a  pilot  test.  To  aid  in  the  development  of  such  a 
pilot  test,  Congress  could  require  the  Secretary  of  Veterans  Affairs  to 
propose  options  that  would  allow  the  agency  the  flexibility  to  test  and 
refine  the  new  process  in  a  cost-effective  and  efficient  manner,  while 
ensuring  pre-established  interim  goals  and  success  criteria  are  being  met 
prior  to  full  implementation. 
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VA  concurred  in  principle  with  the  draft  recommendation  that  it  finalize  a 
detailed  workforce  plan  that  includes  steps  for  training,  support,  and  risk 
mitigation  strategies.  VA  noted  that,  in  addition  to  currently  implementing 
a  fiscal  year  2017  workforce  plan  to  hire  additional  staff,  as  discussed  in 
the  report,  among  other  efforts  it  has  recently  launched  new  attorney 
training  and  continues  to  collaborate  across  the  agency  to  identify  space 
where  new  staff  can  be  located.  We  have  incorporated  these  updates  into 
our  report,  as  appropriate.  In  light  of  these  efforts,  and  because  future 
steps,  such  as  developing  training  materials  on  the  new  appeals  process, 
are  contingent  upon  appeals  reform  legislation,  VA  stated  it  considers  this 
recommendation  complete  and  requested  closure.  While  we  recognize 
that  VA  has  made  progress  and  that  certain  actions,  such  as  training  on  a 
new  process,  is  contingent  upon  reform  legislation,  we  disagree  that  the 
recommendation  should  be  closed.  As  noted  in  our  report,  we  found  that 
VA’s  final  recruiting,  hiring,  and  training  plans  lacked  important  details. 

For  example,  VA  officials  were  still  updating  training  curriculum  that 
supports  work  conducted  in  a  virtual  environment  and  which  is  critical  for 
managing  space  restrictions  for  new  staff.62  Without  a  detailed  workforce 
plan  in  place,  VA  cannot  assess  the  success  of  its  human  capital 
approach,  maximize  its  investments,  or  fully  mitigate  risks.  More  detailed 
workforce  plans  would  help  VA  avoid  the  risks  that  staff  will  not  be  hired 
in  time,  not  be  properly  trained,  or  not  have  the  support  necessary  to 
process  appeals.  Waiting  until  legislation  is  enacted  magnifies  these 
risks.  We  believe  additional  action  is  needed  to  meet  the  intent  of  this 
recommendation;  we  also  clarified  the  recommendation  language  to  state 
that  VA  needs  a  more  detailed  plan. 

VA  concurred  in  principle  with  our  recommendation  that  it  develop  a 
schedule  for  IT  updates  that  lays  out  when  and  how  any  process  reforms 
will  be  integrated  into  its  Caseflow  system.  More  specifically,  VA  noted 
that  it  will  rely  on  the  agile  process  to  develop  Caseflow — whereby  new 
functions  are  continually  added  to  the  system  as  new  user  needs  or  policy 
changes  arise — and  does  not  plan  to  define  schedules  beyond  6  months. 
Given  that  Caseflow  development  related  to  the  new  appeals  process  is 
dependent  on  the  enactment  of  new  legislation,  VA  stated  it  considers 
this  recommendation  to  be  complete.  While  it  is  true  that  the  agile 
process  can  help  mitigate  risks  and  avoid  cost  overruns  and  delays,  we 


62ln  addition,  as  of  December  201 6,  key  aspects  of  its  hiring  plan  were  still  underway.  For 
example,  the  Board  stated  that  as  part  of  its  mitigation  plan  for  ensuring  sufficient  space 
for  new  hires  in  2017,  it  was  in  the  process  of  reopening  negotiations  with  the  union  to 
allow  attorneys  to  work  remotely,  and  planned  to  work  with  the  Office  of  Administration  to 
identify  additional  opportunities  to  use  space  more  efficiently. 
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do  not  believe  this  approach  precludes  VA  from  taking  additional  steps  to 
consider  the  scope  of  potential  changes  required  by  a  new  appeals 
process  and  have  a  broad  plan  in  place  to  ensure  that  all  aspects  of  the 
new  process  are  adequately  supported  by  Caseflow.  We  believe  it  is 
especially  important  for  VA  to  have  specific  time  frames  for  completing 
Caseflow  considering  the  scope  of  the  changes  being  proposed. 
Moreover,  VA  noted  that  components  of  Caseflow  developed  so  far  will 
not  need  to  be  significantly  changed  because  of  appeals  reform 
legislation  being  enacted,  but  VA  did  not  provide  documentation  to 
support  this  assertion.  In  light  of  these  issues,  we  believe  VA  has  not  yet 
met  the  intent  of  this  recommendation. 

VA  concurred  in  principle  with  our  recommendation  that  it  should  conduct 
additional  sensitivity  analyses  around  the  assumptions  used  in  its  models. 
VA  noted  that  sensitivity  analyses  are  valuable  and  that  it  has  focused  its 
efforts  on  risks  its  staff  identified  as  most  likely,  such  as  variations  in 
staffing  and  productivity,  and  the  effect  of  remands.  VA  stated  it  would 
continue  to  analyze,  update,  and  refine  its  modeling,  and  considers  this 
recommendation  to  be  complete.  While  we  recognize  the  logic  of  focusing 
modeling  resources  on  key  variables,  VA  did  not  fully  examine  three  of 
the  four  hiring  surge  options  it  proposed.  Moreover,  VA  did  not  assess  the 
compound  effect  that  would  result  in  changing  multiple  assumptions  at 
once.  Given  the  complexity  of  proposed  changes  and  the  number  of 
variables  beyond  VA’s  control,  we  believe  that  additional  analyses  are 
needed  to  identify  potential  risks  that  may  warrant  additional  mitigation 
strategies.  In  addition,  if  VA  goes  forward  with  appeals  process  reform 
and  begins  to  collect  real-time  data,  these  data  could  improve  modeling 
accuracy  and  serve  as  a  valuable  management  tool. 

VA  concurred  in  principle  with  our  recommendation  to  develop  a  more 
robust  plan  for  closely  monitoring  the  implementation  of  its  process 
reform,  that  includes  metrics  and  interim  goals  to  help  VA  track  progress, 
evaluate  efficiency  and  effectiveness,  and  pinpoint  trouble  spots.  VA 
agreed  that  developing  such  a  plan  is  valuable  for  monitoring  the 
implementation  of  process  reform,  and  should  include  metrics  and  interim 
goals.  However,  VA  stated  that  it  considers  this  recommendation 
complete,  noting  that  preparing  such  a  detailed  plan  depends  on  appeals 
reform  legislation  being  enacted,  and  that  it  will  incorporate  specific  goals 
and  metrics  as  it  moves  towards  implementation.  While  we  recognize  that 
VA  cannot  assume  to  know  the  exact  provisions  that  may  be  included  in 
future  enacted  legislation,  nor  can  it  predict  when  appeals  reform  might 
be  enacted,  we  consider  having  a  more  robust  monitoring  plan  to  be 
essential  to  the  successful  implementation  of  a  new  appeals  process. 
Moreover,  the  absence  of  such  a  plan  raises  questions  as  to  how  VA  will 
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ensure  appropriate  resources  are  devoted  to  managing  appeals  under  the 
new  versus  old  process,  or  intended  results  are  achieved  as  the  new 
process  is  implemented. 

VA  concurred  in  principle  with  our  recommendation  to  develop  a  strategy 
to  transparently  report  to  Congress  and  the  public  on  veterans’ 
experiences  with  the  new  appeals  process.  VA  noted  that  it  is  already 
developing  timeliness  goals  for  three  of  the  four  appeal  options  in  the 
proposed  new  process,  as  discussed  in  our  report.  VA  said  it  also  plans 
to  measure  success  of  the  new  process  with  results  from  customer 
satisfaction  surveys  and  is  developing  a  dashboard  for  internal 
performance  monitoring.  VA  did  not  agree  that  measuring  overall  appeals 
resolution  timeliness  is  an  appropriate  a  measure  and  believes  tracking 
time  frames  for  each  of  the  options  separately  is  more  appropriate.  While 
we  agree  that  metrics  based  on  the  different  options  could  be  valuable  for 
VA,  the  Congress,  and  the  public,  we  disagree  that  VA’s  focus  on 
measuring  timeliness  by  option  is  in  the  best  interest  of  the  veteran. 
Because  veterans  may  pursue  more  than  one  option  under  VBA,  the 
Board  or  both,  we  believe  that  VA’s  approach  does  not  take  into  account 
the  veteran’s  perspective  of  how  long  it  took  for  them  to  receive  a  final 
appeal  decision.  Metrics  from  the  veterans’  overall  perspective  would 
complement,  not  replace,  metrics  for  VBA,  the  Board,  and  each  option. 
Further,  because  VA’s  approach  does  not  allow  VA  to  compare  the  new 
process  with  the  old  or  to  determine  whether  the  new  process  represents 
an  improvement  over  the  old  process,  we  believe  it  does  not  promote 
transparency  in  reporting  to  the  Congress  and  the  public. 
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We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Secretary  of  Veterans  Affairs,  the  Under  Secretary  for 
Benefits,  the  Chairman  of  the  Board  of  Veterans  Appeals,  and  the  VA’s 
Chief  Information  Officer.  In  addition,  the  report  is  available  at  no  charge 
on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)  512-7215  or  bertonid@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  GAO  staff  who  made  key  contributions  to  this 
report  are  listed  in  appendix  II. 


Daniel  Bertoni 
Director 

Education,  Workforce,  and  Income  Security  Issues 
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Chairman 
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Ranking  Member 
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United  States  Senate 

The  Honorable  Jerry  Moran 
Chairman 

The  Honorable  Brian  Schatz 
Ranking  Member 

Subcommittee  on  Military  Construction,  Veterans  Affairs,  and  Related 
Agencies 

Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Phil  Roe 
Chairman 
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The  Honorable  Mike  Bost 
Chairman 

The  Honorable  Elizabeth  Esty 
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Appendix  I:  Comments  from  the  Department 
of  Veterans  Affairs 


Department  of  Veterans  Affairs 
Washington  DC  20420 

February  27,  2017 


Mr.  Daniel  Bertoni 
Director,  Education,  Workforce, 
and  Income  Security  Issues 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Mr.  Bertoni: 

The  Department  of  Veterans  Affairs  (VA)  has  reviewed  the  Government 
Accountability  Office’s  (GAO)  draft  report,  “VA  DISABILITY  BENEFITS:  Additional 
Planning  Would  Enhance  Efforts  to  Improve  the  Timeliness  of  Appeals  Decisions” 
(GAO-17-234). 

The  enclosure  provides  our  general  and  technical  comments  and  sets  forth  the 
actions  to  be  taken  to  address  the  GAO  draft  report  recommendations. 

VA  appreciates  the  opportunity  to  comment  on  your  draft  report. 


Sincerely, 


Gina  S.  Farrisee 
Deputy  Chief  of  Staff 


Enclosure 
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General  Comments: 

The  Department  of  Veterans  Affairs  (VA)  appreciates  the  work  of  the  Government 
Accountability  Office  (GAO)  in  evaluating  VA’s  approaches  to  addressing  challenges 
that  are  impacting  the  timeliness  of  VA  appeals  decisions.  VA  is  grateful  for  GAO’s 
recognition  that  VA  has  taken  steps  to  bolster  capacity  and  improve  the  efficiency  and 
effectiveness  of  VA’s  disability  appeals  process  and  GAO’s  acknowledgement  that  is  in 
line  with  sound  planning  practices  were  employed.  However,  VA  disagrees  with  GAO’s 
conclusion  that  VA’s  plans  do  not  account  for  significant  remaining  changes.  GAO  is 
correct  to  describe  VA’s  proposed  appeals  reform  as  ambitious,  but  nothing  less  than 
ambitious,  comprehensive,  legislative  reform  will  adequately  modernize  the  current, 
broken  appeals  process  and  enable  VA  to  provide  the  best  service  to  Veterans. 

GAO  recommends  pilot  testing  appeals  process  reform,  in  order  to  manage  risks; 
however,  GAO  also  states  in  its  report  that  pilot  testing  is  not  required  in  all 
circumstances,  only  that  it  is  one  of  a  number  of  methods  that  can  be  utilized  to  assess 
the  costs,  benefits,  and  feasibility  of  an  alternative  process.  GAO  has  also 
acknowledged  that  pilot  testing  is  not  the  only  method  of  achieving  risk  mitigation  goals. 
As  reported,  VA  has  utilized  modeling,  one  of  the  methods  identified  as  a  sound 
redesign  practice,  to  project  costs  and  benefits  of  the  new  process.  VA  is  in  the  process 
of  updating  and  validating  its  model,  and  has  advised  GAO  that  it  will  continue  to  update 
the  model  as  additional  information,  such  as  actual  production  data,  becomes  available. 
GAO  has  expressed  concern  that  VA  may  be  exposing  itself  to  unforeseen  risks  and 
setbacks  that  could  slow  progress  toward  improving  appeals  decision  timeliness. 
However,  VA  has  carefully  assessed  the  potential  risks  and  concluded  that  they  are 
manageable  and  that  the  far  greater  risk  for  Veterans  would  be  failure  to  enact 
legislative  appeals  reform.  Further,  VA  has  developed  mitigation  strategies  for  each 
risk,  which  were  shared  with  GAO.  Importantly,  as  also  previously  stated  to  GAO, 
several  of  the  potential  risks  identified,  which  VA  is  actively  engaged  in  mitigating,  are 
not  specifically  related  to  the  new  process  itself,  but  are  related  to  actions  VA  is  taking 
irrespective  of  legislative  reform,  for  example,  hiring  additional  full-time  equivalent  (FTE) 
employees  and  making  information  technology  (IT)  improvements.  VA  does  not  agree 
that  any  appeals  process  reform  piloting  is  necessary,  particularly  when  it  has  proposed 
an  18-month  implementation  period  and  the  broad  consensus  for  change  will  allow  for 
post-implementation  improvements  as  they  become  necessary. 

GAO  states  in  its  report  that  VA  previously  supported  pilot  testing  a  new  appeals 
process.  It  is  correct  that  VA  did  generally  support  the  Fully  Developed  Appeals  (FDA) 
pilot  program  in  congressional  bill  H.R.  800  as  reflected  in  the  April  14,  2015,  testimony 
before  the  House  Committee  on  Veterans’  Affairs,  Subcommittee  on  Disability 
Assistance  and  Memorial  Affairs.  However,  VA  subsequently  clarified  that  the  growing 
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appeals  challenge  requires  much  more  widespread  reform,  in  VA’s  March  15,  2016, 
testimony  to  the  Senate  Committee  on  Veterans’  Affairs,  Secretary  McDonald  stated: 

“For  this  hearing,  the  Committee  has  identified  a  bill,  S.  2473,  the  Express 
Appeals  Act,  which  would  establish  a  pilot  program  for  what  are  called  Fully 
Developed  Appeals  (FDA),  which  would  limit  new  evidence  filed  after  the  point  of 
appeal  through  a  voluntary  program.  VA  has  supported  the  FDA  pilot  in  the  past, 
but  at  this  point,  we  believe  the  growing  appeals  challenge  requires  much  more 
widespread  reform  that  will  address  all  future  appeals,  not  just  the  voluntary 
participants  that  may  elect  the  FDA  pilot.  S.  2473  will  not  reduce  the  pending 
appeals  inventory  and  will  not  significantly  address  the  future  appeals  inventory. 
As  a  pilot  for  voluntary  participants,  we  believe  it  does  too  little  to  streamline  the 
VA  appeals  process  for  all  Veterans,  or  to  provide  an  improved  experience  for  all 
Veterans.  The  current  VA  appeals  process  is  lengthy,  complex,  confusing,  and 
frustrating  for  Veterans.  All  Veterans,  not  just  those  who  elect  to  participate  in  an 
optional  FDA  pilot  program,  deserve  an  efficient,  transparent,  and  streamlined 
appeals  experience.  The  FDA  pilot  program  in  its  current  form  is  not  enough  to 
change  the  current  broken  VA  appeals  system.  True  comprehensive  legislative 
reform  that  is  as  ambitious  as  that  presented  in  the  President’s  2017  Budget  is 
required.” 

VA  remains  committed  to  its  position  that  pilot  testing,  or  any  type  of  pilot  program,  will 
not  provide  the  widespread  reform  needed.  GAO  incorrectly  states  that  VA  reached  the 
decision  that  the  current  framework  is  so  fundamentally  broken,  and  proposed  the  new 
appeals  process  as  the  better  option  without  analyzing  feasible  alternatives.  VA 
designed  the  new  appeals  process  based  on  the  collective  experience  of  internal  and 
external  experts  who  have  worked  for  years  in  the  current  appeals  process.  The  fact 
that  so  many  experts  were  able  to  come  to  a  consensus  regarding  a  new  design  that 
will  be  beneficial  to  Veterans,  Veterans’  representatives,  the  agency,  and  taxpayers, 
should  not  be  taken  lightly  and  supports  the  position  that  the  new  appeals  process 
should  be  enacted  and  implemented  as  comprehensive  and  permanent  reform.  VA  and 
its  stakeholders  are  confident  in  this  design,  and  support  enactment  in  full.  Not  only  is 
the  proposed  new  appeals  process  strongly  supported  by  its  stakeholders,  but  it  has 
congressional  support  as  well.  If  this  widespread  reform  is  not  enacted,  the  problem  will 
only  continue  to  get  worse,  as  more  and  more  appeals  will  enter  the  current  appeals 
process,  a  process  that  is  complex,  inefficient,  ineffective,  confusing,  frustrating,  and 
takes  too  long. 

GAO  also  indicates  that  VA’s  plans  for  hiring  staff  and  upgrading  IT  lacks  key  details 
and  VA  lacks  a  robust  monitoring  plan.  VA  has  developed,  and  shared  with  GAO,  a 
comprehensive  implementation  plan,  which  was  developed  collaboratively  by  the  Board 
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of  Veterans’  Appeals  (Board)  and  Veterans  Benefits  Administration  (VBA).  VA  has  also 
shared  more  specific  plans  regarding  hiring,  space  for  new  FTE,  recruitment,  and 
training. 

Regarding  GAO’s  position  that  VA’s  plans  for  upgrading  IT  in  support  of  process  reform 
lack  key  details,  VA  has  described  how  the  agile  software  development  methods  used 
by  VA’s  Office  of  Information  and  Technology  Digital  Service  (DS)  proscribe  the  creation 
of  comprehensive  requirements  prior  to  initiating  development.  These  methods  are 
common  practice  for  leading  technology  companies  in  the  private  sector  and  are 
consistent  with  GAO-12-681 ,  Software  Development:  Effective  Practices  and 
Federal  Challenges  in  Applying  Agile  Methods.  The  agile  approach  used  by  DS  in 
building  the  new  Caseflow  system  works  in  two-week  iterations  or  “sprints,"  and  new 
code  is  continually  integrated  into  the  production  environment,  meaning  that 
improvements  are  made  available  to  users  as  frequently  as  every  day.  The  contents  of 
each  sprint  are  informed  by  a  schedule  that  extends  six  months  into  the  future  and  is 
continually  revised  in  response  to  availability  of  new  information  and  changes  in  policy, 
law,  and  regulation,  including  the  proposed  process  reform.  Caseflow  does  not  include 
any  component  that  would  need  to  undergo  significant  changes  as  a  result  of  process 
reform,  and  upon  passage,  DS  would  accordingly  prioritize  for  development 
components  adjacent  to  areas  impacted  by  the  reform  in  order  to  meet  the  timeline  for 
rollout  of  the  new  process. 

VA  is  also  prepared  to  measure  success  of  the  new  appeals  process.  As  stated  to 
GAO,  VA  recognizes  the  potential  risk  associated  with  not  being  able  to  accurately  track 
metrics  in  the  new  appeals  process,  as,  without  realistic  and  useful  metrics,  and  reliable 
data,  VA  will  not  be  able  to  objectively  ascertain  the  success  of  the  new  appeals 
process.  However,  VA  has  also  described  the  ways  that  it  will  measure  metrics  related 
to  the  new  appeals  process  following  passage  of  the  legislation.  GAO  states  that  it 
remains  to  be  determined  how  and  whether  VA  will  be  able  to  measure  the  extent  to 
which  the  new  process  reduces  “churning”  compared  to  the  old  process.  However,  the 
new  appeals  process,  as  proposed,  eliminates  “churning”  by  design,  as  it  removes  the 
multiple  steps  and  split  jurisdiction  of  appeals  processing  in  which  the  same  issue  is 
adjudicated  multiple  times  by  various  offices  with  no  finality.  The  new  appeals  process 
is  designed  so  that  each  lane  has  a  clearly  defined  start/end  point. 

Overall,  VA  has  not  identified  any  risk  that  would  justify  pilot  testing.  VA  has  developed 
comprehensive  plans  to  implement  the  new  appeals  process,  to  hire  and  train  additional 
staff,  and  to  update  IT  systems  that  will  facilitate  successful  implementation  and 
execution  in  each  of  these  areas.  Further,  VA  has  developed  plans  to  track  and  monitor 
progress,  once  appeals  reform  legislation  is  implemented,  and  has  shared  those  plans 
with  GAO.  VA  has  not  only  worked  collaboratively  with  stakeholders  to  design  and 
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propose  a  new  appeals  process  that  will  allow  VA  to  best  serve  Veterans,  but  it  has 
undertaken  necessary  planning  efforts  to  put  itself  in  a  position  to  successfully 
implement  this  process  change  concurrently  with  other  ongoing  efforts,  to  specifically 
include  hiring  and  IT  updates,  VA  has  extensive  experience  with  leading  organizational 
change  management,  having  implemented  high-impact  people,  process,  and 
technology  initiatives,  which  have  completely  transformed  a  complex,  paper-bound, 
compensation  claims  process  into  a  lean,  digital,  and  semi-automated  process.  The 
Department  has  gathered  indispensable  best  practices  that  it  can  draw  upon  when 
implementing  large-scale  change  such  as  the  appeals  reform  legislation. 
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GAO  Recommendation:  To  further  align  efforts  to  address  appeals  workload  and 
improve  timeliness  of  decisions,  and  reduce  the  risk  that  efforts  will  not  go  as 
planned,  we  recommend  that  the  Secretary  of  Veterans  Affairs  direct  the  Under 
Secretary  for  Benefits;  the  Chairman,  Board  of  Veterans’  Appeals;  and  the  Chief 
Information  Officer,  as  appropriate,  to: 

Recommendation  1:  Incorporate  pilot  testing  of  the  proposed  process  into 
implementation  plans,  and  pursue  necessary  legislative  authority. 

VA  Comment:  Non-concur.  There  is  broad  consensus  that  the  current  Department  of 
Veterans  Affairs  (VA)  appeals  system  is  broken  and  in  urgent  need  of  reform  and  that 
the  proposed  process,  which  was  designed  by  VA,  Veterans  Service  Organizations 
(VSOs)  and  other  stakeholders  (“new  appeals  process")  is  a  better  and  less  costly 
design.  Anything  short  of  full  implementation  on  the  proposed  delayed  effective  date 
(18  months  after  passage  of  the  legislation),  would  be  disastrous  for  Veterans.  The  new 
appeals  process  must  be  fully  implemented  to  end  the  flow  of  appeals  into  the  legacy 
system,  which  is  bad  for  Veterans.  While  a  pilot  program  is  often  useful  for  testing 
changes  to  discrete  elements  of  an  existing  program  or  process,  such  as  VA's 
implementation  of  a  standard  Notice  of  Disagreement  form  in  its  current  appeals 
process,  it  is  not  useful  for  wholescale  reforms  to  Veterans’  fundamental  rights,  such  as 
providing  Veterans  an  entirely  new  appeals  process.  Moreover,  it  would  be 
unreasonable  for  VA  to  pursue  legislation  authorizing  a  pilot  program  when  it  has  more 
than  469,000  appeals  pending,  the  average  Veteran  is  waiting  years  for  a  decision,  and 
VA  and  its  stakeholders  have  concluded  publicly  that  the  current  process  is  harming 
Veterans.  Pursuing  a  pilot  program  would  suggest  that  keeping  the  current  process  or 
delaying  implementation  of  a  known  better  design,  would  somehow  be  acceptable  to  the 
Veterans  the  Department  serves.  VA  does  not  agree  that  any  such  piloting  is 
necessary,  particularly  when  it  has  proposed  an  18-month  implementation  period  and 
the  broad  consensus  for  change  will  allow  for  post-implementation  improvements  as 
they  become  necessary. 

GAO  acknowledges  that  pilot  testing  prior  to  full  implementation  would  delay  the 
reforms  that  Veterans  so  desperately  need.  This  is  not  acceptable  for  Veterans,  the 
Department,  or  taxpayers.  The  longer  it  takes  to  enact  appeals  reform,  the  longer 
Veterans  will  have  to  wait  for  a  decision  on  their  appeal  as  more  and  more  appeals 
enter  the  current,  broken  system,  and  the  longer  taxpayers  will  have  to  fund  a  broken, 
inefficient,  and  costly  system.  If  the  appeals  reform  legislation  is  not  passed,  the 
problem  is  only  going  to  get  worse. 

VA  is  also  concerned  that  piloting  the  new  appeals  process  for  some  Veterans  would 
deprive  similarly  situated  Veterans  of  the  liberalized  effective  date  and  early  resolution 
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provisions  in  the  proposed  new  appeals  process,  and  thus  would  be  perceived  as 
inequitable.  Even  if  Congress  were  to  authorize  a  pilot  that  favored  certain  Veterans, 
the  authorizing  statute  would  raise  constitutional  issues  and  prompt  litigation.  Given  the 
broad  consensus  on  the  need  for  change  and  the  strong  stakeholder  support  for  the 
new  appeals  process  as  the  solution,  VA  sees  no  benefit  in  pursuing  an  option  that 
might  raise  legal  concerns. 

GAO  notes  that  sound  redesign  practices  suggest  using  cost-effective  methods,  such 
as  prototyping,  modeling  and/or  computer  simulation,  or  limited  pilot  testing,  to  assess 
the  costs  and  benefits  to  inform  the  selection  of  a  feasible  alternative  process  with  a 
high  return  on  investment.  GAO  does  not  indicate  that  pilot  testing  is  required  or  is  a 
best  practice  in  all  circumstances,  only  that  it  is  one  of  a  number  of  methods  that  can  be 
utilized  to  assess  the  costs,  benefits,  and  feasibility  of  an  alternative  process,  and  GAO 
acknowledges  that  pilot  testing  is  not  the  only  method  of  achieving  risk  mitigation  goals. 
Significantly,  VA  has  utilized  modeling,  one  of  the  methods  identified  as  a  sound 
redesign  practice,  to  project  costs  and  benefits  of  the  new  appeals  process.  VA  is  in  the 
process  of  updating  and  validating  its  model  and  has  advised  GAO  that  it  will  continue 
to  update  the  model  as  additional  information,  such  as  actual  production  data,  becomes 
available.  VA  is  also  actively  engaged  in  risk  mitigation. 

VA  has  acknowledged  the  importance  of  risk  mitigation  and  has  presented  GAO  with 
several  potential  risks,  mitigation  strategies  for  each,  and  expected  dates  of  mitigation. 
Specifically,  VA  has  addressed  and  shared  with  GAO  risks  related  to  the  process, 
staffing  and  space,  information  technology  (IT),  the  legacy  inventory,  implementation, 
modeling  assumptions,  and  metrics.  Even  considering  the  risks  in  each  of  these  areas, 
VA  has  concluded  that  the  far  greater  risk  for  Veterans  would  be  failure  to  enact 
legislative  appeals  reform.  This  determination  is  based  on  the  fact  that  the  potential 
risks  are  manageable,  and  VA  has  mitigation  strategies  for  each.  It  is  also  important  to 
note  that  several  of  these  risks  are  not  related  to  the  legislation  itself  and,  therefore, 
piloting  the  new  process  would  not  be  helpful  in  reducing/eliminating  risk.  Specifically, 
VA  has  already  hired  additional  employees  and  continues  to  address  process  and  IT 
improvements  regardless  of  whether  or  not  Congress  acts  to  provide  Veterans  the 
legislative  appeals  reform  they  deserve.  VA  has  not  identified  any  risk  that  would  justify 
pilot  testing  the  new  appeals  process  legislation. 

Further,  VA  has  a  strong,  comprehensive  implementation  plan  in  place,  carefully  and 
collaboratively  developed  by  the  Board  and  VBA,  drawing  on  expertise  of  agency  and 
stakeholder  subject  matter  experts,  and  can,  therefore,  successfully  implement  the  new 
appeals  process  within  the  18-month  period  it  proposed.  VA  has  extensive  experience 
with  leading  organizational  change  management  in  its  organization,  having 
implemented  high-impact  people,  process,  and  technology  initiatives,  which  have 
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completely  transformed  a  complex,  paper-bound,  compensation  claims  process  into  a 
lean,  digital,  and  semi-automated  process.  No  one  can  deny  the  successful  outcome  of 
that  huge  transformational  change.  As  a  result,  VA  has  learned  many  valuable  lessons 
that  it  can  trust  regarding  how  to  best  implement  large-scale  change  and  has  gathered 
indispensable  best  practices  that  will  be  drawn  upon  when  implementing  the  appeals 
reform  legislation. 

Regarding  opportunities  to  correct  problems  with  the  new  design  and  refine  the  process 
prior  to  full  implementation,  pilot  testing  would  not  be  beneficial.  Changes  to  the  current 
VA  appeals  process  require  legislative  action.  VA  does  not  foresee  changes  needed  in 
the  design.  However,  if  VA  discovers  that  such  revision  is  warranted,  additional 
legislative  action  would  be  required  to  make  such  change,  whether  the  new  appeals 
process  is  initially  enacted  in  full  or  as  a  pilot. 

Overall,  VA  disagrees  with  GAO’s  recommendation.  The  new  appeals  process  was 
designed  based  on  the  collective  experience  of  experts  who  have  worked  for  years  in 
the  current  appeals  framework.  The  fact  that  so  many  experts  were  able  to  reach  a 
consensus  regarding  a  new  design  that  will  be  beneficial  to  Veterans,  Veterans’ 
representatives,  the  agency,  and  taxpayers  should  not  be  taken  lightly,  and  supports  the 
position  that  the  new  appeals  process  should  be  enacted  and  implemented  as  a 
comprehensive  and  permanent  reform.  VA,  VSOs,  and  other  stakeholders  are 
confident  in  this  design,  and  support  enactment  in  full.  Not  only  is  the  new  appeals 
process  supported  by  both  internal  and  external  stakeholders,  but  it  has  congressional 
support  as  well.  Legislation  to  reform  the  VA  appeals  process  was  introduced  in  four 
bills  during  the  1 14th  Congress,  with  H.R.  5620  passing  the  House,  and  has  been 
reintroduced  in  three  bills  in  the  1 1 5th  Congress.  This  legislative  action  is  a  testament  to 
support  for  the  new  appeals  process. 
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Recommendation  2:  Finalize  a  detailed  workforce  plan,  that  includes  steps  for  (1) 
training,  (2)  support  (such  as  office  space  and  telework  guidance)  to  prepare  for 
and  maximize  the  effectiveness  of  additional  staff  resources,  and  (3)  risk 
mitigation  strategies  that  consider  how  the  timing  of  recruitment  and  training 
dovetails  with  uncertain  timeframes  for  implementing  a  new  appeals  process. 

VA  Comment:  Concur  in  principle.  VA  has  already  developed  plans  regarding  hiring, 
training,  support  (space  and  telework),  as  well  as  mitigation  strategies.  On  October  28, 
2016,  VA  provided  GAO  plans  regarding  hiring  and  space,  as  well  as  a  document 
describing  risks  and  mitigation  strategies.  The  Board  provided  additional  information 
regarding  recruiting  on  October  31,  2016  and  provided  additional  information  regarding 
training  on  November  14  -  15,  2016.  More  information  addressing  risks  and  mitigation 
strategies,  including  risks  associated  with  staffing  and  space,  were  provided  to  GAO  on 
December  16,  2016. 

Specifically,  the  Board  is  currently  implementing  a  fiscal  year  (FY)  2017  workforce  plan 
to  hire  and  onboard  an  additional  242  FTEs  for  a  total  of  922  cumulative  FTEs  based  on 
its  FY  2017  funding.  As  of  January  31,  2017,  the  Board  had  738  cumulative  FTEs.  The 
Board  also  launched  new  attorney  training,  which  consists  of  four  weeks  of  new 
attorney  “Bootcamp”  provided  by  the  Board's  Office  of  Knowledge  Management  and 
eight  weeks  of  one-on-one  mentoring  by  experienced  attorneys.  In  order  to 
accommodate  additional  FTEs,  the  Board  is  in  the  process  of  reconfiguring  existing 
space  to  accommodate  additional  workstations  and  has  approved  a  new  remote 
telework  program,  which  will  allow  attorneys  to  report  to  the  Board’s  offices  four  times  a 
year  and  make  additional  workstation  spaces  available.  The  Board  has  also  obtained 
hoteling  software  to  facilitate  reservations  of  workstations  for  teleworking  personnel  on 
days  they  report  to  the  office  and  allow  for  efficient  tracking  of  available  workstations  at 
the  Board.  The  Board  recognizes  the  risk  that  failure  to  successfully  hire  will  impact  the 
number  of  appeals  decisions  the  Board  issues  to  Veterans  and  the  risk  associated  with 
the  challenges  of  providing  adequate  space  for  the  additional  FTEs  hired  in  FY  2017, 
and  is  actively  engaged  in  mitigating  these  risks.  Specifically,  the  Center  of  Excellence 
Pilot  Program  for  hiring  legal  professionals  was  established  with  VA’s  Office  of  Fluman 
Resources  &  Administration  to  support  the  Board’s  aggressive  hiring  and  the  Board  is 
reconfiguring  existing  space,  utilizing  telework,  and  collaborating  with  VA’s  Office  of 
Administration  to  identify  space  opportunities. 

As  indicated  above,  the  Board  is  implementing  its  FY  2017  workforce  plan.  Since  a 
more  detailed  workforce  plan  is  highly  dependent  on  VA's  annual  budget  appropriation, 
the  Board  will  update  its  workforce  plan  for  the  future.  That  updated  plan  will 
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incorporate  steps  for  training  and  address  support  (including  office  space  and  telework 
guidance).  VA  considers  this  recommendation  complete  and  requests  closure. 
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Recommendation  3:  Develop  a  schedule  for  IT  updates  that  explicitly  addresses 
when  and  how  any  process  reform  will  be  integrated  into  new  systems  and  when 
Caseflow  will  be  ready  to  support  a  potential  streamlined  appeals  process  at  its 
onset. 

VA  Comment:  Concur  in  principle.  VA  recognizes  the  importance  of  effective  project 
management  practices  for  the  development  of  information  systems.  Such  practices 
should  anticipate  new  information  and  requirements,  and  accordingly,  the  software  team 
developing  the  new  Caseflow  system  is  ready  to  adapt  its  development  schedule  to  a 
streamlined  appeals  process  and  integrate  and  support  process  reform  at  its  onset. 

As  is  the  practice  of  agency  teams  of  the  United  States  Digital  Service,  and  common  of 
leading  private  sector  technology  companies,  the  Caseflow  team  implements  agile 
software  development  methods  consistent  with  GAO-12-681,  Software  Development- 
Effective  Practices  and  Federal  Challenges  in  Applying  Agile  Methods.  In  contrast 
to  traditional  waterfall  software  development,  the  team  does  not  develop  a 
comprehensive  set  of  requirements  prior  to  initiating  development,  but  rather  develops 
the  project  in  two-week  iterations  or  "sprints."  New  code  is  continually  integrated  into 
the  production  environment,  meaning  that  improvements  are  made  available  to  users  as 
frequently  as  daily.  The  contents  of  each  sprint  are  informed  by  a  schedule  that 
extends  six  months  into  the  future  and  is  continually  revised  to  reflect  new  information 
and  shifts  in  external  factors. 

These  agile  practices  offer  specific  advantages  in  the  context  that  the  Caseflow  team 
operates.  The  team  has  deliberately  chosen  not  to  define  schedules  beyond  six  months 
in  order  to  evolve  requirements  in  response  to:  1)  availability  of  new  research  and 
analysis  of  improved  baseline  data;  2)  changes  in  Board  staffing  and  policy  and  the 
effects  thereof;  and  3)  changes  to  the  legal  and  regulatory  environment,  including  the 
proposed  reform. 

The  team  engages  in  a  quarterly  prioritization  exercise  in  coordination  with  Board 
stakeholders,  evaluating  which  of  the  remaining  components  of  the  Caseflow  product 
should  be  incorporated  next,  or  which  of  the  existing  components  should  be  iterated, 
and  how  to  allocate  resources  among  these  various  potential  improvements.  This 
exercise  establishes  milestones  that  the  team  will  work  toward  while  planning  each 
two-week  sprint.  In  addition  to  evaluating  whether  and  to  what  extent  these 
improvements  serve  the  team's  goal  to  "empower  employees  with  technology  to 
increase  timely,  accurate  appeals  decisions  and  improve  the  Veteran  experience,"  and 
to  reduce  the  Board’s  dependency  on  and  ultimately  deprecate  the  Veterans  Appeals 
Control  and  Locator  System  (VACOLS),  this  exercise  considers  the  known  potential  for 
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the  three  external  factors  listed  above  to  affect  the  requirements,  utility,  and  longevity  of 
the  product  as  developed. 

To  date,  the  Caseflow  team  has  not  developed  any  component  that  would  need  to 
undergo  significant  changes  as  a  result  of  the  proposed  legislative  process  reform. 
Should  Congress  pass  such  reform,  the  team  expects  that  components  adjacent  to  the 
areas  impacted  by  the  reform  would  be  accordingly  prioritized  for  development  in  order 
to  meet  the  timeline  for  rollout  of  the  new  process.  Given  the  uncertainty  of  the  timeline 
for  adopting  the  proposed  reforms,  to  date,  the  team  has  not  prioritized  these 
components.  Given  that  Caseflow  development  related  to  the  new  appeals  framework 
is  highly  dependent  on  enactment  of  the  appeals  reform  legislation,  VA  considers  this 
recommendation  complete  and  requests  closure. 

Recommendation  4:  Conduct  additional  sensitivity  analyses  around  the 
assumptions  used  in  projection  models  to  more  accurately  estimate  future 
appeals  inventories  and  timeliness.  In  doing  so,  consider  running  additional 
analyses  on  how  these  factors  in  conjunction  with  one  another  may  affect  the 
timeliness  and  cost  of  deciding  pending  appeals. 

VA  Comment:  Concur  in  principle.  In  conjunction  with  the  model  shared  with  GAO,  VA 
has  already  conducted  several  sensitivity  analyses  around  factors  that  could  be  subject 
to  variation.  As  noted  in  VA’s  October  28,  2016,  response  to  questions  from  GAO, 
“[sjensitivity  analysis  can  [be]  and  was  conducted  within  the  Excel  framework  on  various 
assumptions,  including  number  of  FTEs,  influence  of  remands,  and  organization 
productivity."  Moreover,  in  response  to  GAO  questions,  VA  further  analyzed  the 
model’s  sensitivity  based  upon  changes  to  Veteran  filing  patterns. 

VA  did  not  run  sensitivity  analyses  on  every  possible  metric.  As  noted  on  page  1 12  of 
GAO’s  Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules,  “sensitivity 
indexes  and  correlation  measures  are  useful  starting  points  for  assessing  the  possible 
magnitude  of  realized  risks,  but  they  have  limited  use  in  prioritizing  risks.”  (Schedule 
Assessment  Guide:  Best  Practices  for  Project  Schedules.  GAO  16-89G,  Washington 
D.C.  December  2015).  Accordingly,  in  conjunction  with  the  model  presented  to  GAO, 
VA  ran  sensitivity  analyses  on  "realized”  or  anticipated  risks;  it  did  not  analyze  metrics 
or  risks  that  were  remote,  speculative,  or  entirely  inconsistent  with  VA  and  stakeholders’ 
experience.  This  analysis  of  anticipated  risks  was  based  on  VA’s  decades  of 
experience  in  the  administrative  appeals  process.  Moreover,  VA  prioritized  risks  and 
measures  that  are  considered  more  likely,  such  as  variations  in  FTEs,  influence  of 
remands,  and  organization  productivity. 
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VA  agrees  that  sensitivity  analysis  is  valuable.  VA  will  continue  to  analyze,  update,  and 
refine  its  modeling  as  it  works  towards  implementation.  VA  considers  this 
recommendation  complete  and  requests  closure. 

Recommendation  5:  Develop  a  more  robust  plan  for  closely  monitoring 
implementation  of  process  reform  that  includes  metrics  and  interim  goals  to  help 
track  progress,  evaluate  efficiency  and  effectiveness,  and  pinpoint  trouble  spots. 

VA  Comment:  Concur  in  principle.  VA  has  already  developed  an  implementation  plan, 
which  was  carefully  and  collaboratively  prepared  by  the  Board  and  VBA  and  provided  to 
GAO  on  October  28,  2016.  This  implementation  plan  was  prepared  by  drawing  on 
expertise  of  numerous  subject  matter  experts,  so  that  the  plan  itself  would  be 
comprehensive  and  include  realistic  estimated  timelines.  Many  elements  of  this  plan 
are  currently  underway  and  VA  fully  anticipates  that  the  steps  required  for 
implementation  will  be  completed  during  the  18-month  implementation  period  that  it 
proposed  for  the  legislation.  VA  agrees  that  a  robust  plan  is  valuable  in  monitoring 
implementation  of  process  reform,  and  that  such  a  plan  should  include  metrics  and 
interim  goals  to  help  track  progress.  The  current  plan  will  allow  VA  to  closely  monitor  its 
implementation  of  the  legislation  and  adapt  as  necessary  to  account  for  changing 
circumstances.  Since  a  more  detailed  implementation  plan  is  highly  dependent  on  the 
enactment  of  the  appeals  reform  legislation,  any  necessary  elaboration,  to  include 
incorporation  of  specific  metrics  and  interim  goals,  will  be  incorporated  into  the 
implementation  plan  as  VA  works  towards  implementation.  VA  considers  this 
recommendation  complete. 

To  better  understand  whether  appeals  process  reform,  in  conjunction  with  other 
efforts,  have  improved  timeliness,  we  recommend  the  agency: 

Recommendation  6:  Develop  a  strategy  for  assessing  process  reform  relative  to 
the  current  process  that  ensures  transparency  in  reporting  to  the  Congress  and 
public  on  the  extent  to  which  VA  is  improving  veterans’  experience  with  its 
disability  appeals  process. 

VA  Comment:  Concur  in  principle.  In  its  report,  GAO  states  that  the  new  timeliness 
measures  that  VA  plans  to  report  to  Congress  and  the  public  lack  transparency  on 
whether  overall  appeals  resolution  timeliness  is  improving  from  the  Veterans’ 
perspective.  GAO  asserts  that  an  Appeals  Resolution  Time  (ART)  measure  would 
provide  a  basis  for  comparing  timeliness  under  the  old  versus  the  new  process,  and 
would  provide  historical  perspectives  on  changes  in  timeliness  from  the  Veteran’s  point 
of  view.  However,  ART  is  not  a  Veteran-centric  metric  nor  is  it  an  appropriate  measure 
for  comparing  processing  timeliness  in  all  lanes  in  the  new  process  versus  processing 
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timeliness  in  the  legacy  appeals  system.  The  new  process  is  designed  so  that  each 
lane  has  a  clearly  defined  start  and  end  point,  as  opposed  to  the  legacy  appeals  system 
with  just  one  open-ended  option.  ART,  as  reported  by  VA  prior  to  FY  2015,  represented 
the  average  length  of  time  it  takes  VA  to  process  an  appeal  from  the  date  a  claimant 
files  a  Notice  of  Disagreement  until  a  case  is  finally  resolved;  whether  the  appeal  is 
resolved  at  VBA  or  at  the  Board.  Flowever,  the  new  process  does  not  split  jurisdiction 
over  appeals  processing  between  VBA  and  the  Board;  therefore,  an  ART  measure 
would  not  be  applicable  for  VBA’s  supplemental  claim  and  higher-level  review  lanes  in 
the  new  process  because  these  are  not  appeal  lanes.  Because  ART  measures  average 
timeliness  in  appeals  processing  for  all  appeals,  including  those  that  are  resolved  prior 
to  going  to  the  Board,  using  this  measure  would  not  be  appropriate  in  the  new  process. 

VA  has  already  developed  a  strategy  for  assessing  process  reform  relative  to  the 
current  process.  Rather  than  utilizing  ART,  which,  as  discussed  above,  would  not  be 
appropriate  in  the  new  process,  VA  plans  to  evaluate  the  new  process  by  examining 
wait  times  for  Veterans,  with  decision  processing  times  meeting  timeliness  goal 
averages  of  125  days  in  the  higher-level  review,  125  days  for  supplemental  claims,  and 
365  days  in  the  Board  non-hearing  option  lane.  In  addition  to  Veteran  wait  times,  VA 
plans  to  measure  success  of  the  new  appeals  process  with  the  results  from  customer 
satisfaction  surveys.  VA  believes  that  wait  times  and  results  from  customer  satisfaction 
surveys  are  the  most  adequate  Veteran-centric  metrics  to  capture  success  of  the  new 
process  and  to  compare  to  the  legacy  process.  VA  will  also  continue  to  respond  to 
congressional  tracking  reports,  requests  for  information,  and  questions  for  the  record 
from  Congress,  including  any  questions  regarding  the  extent  to  which  VA  is  improving 
Veterans’  experience  with  its  appeals  process.  Internally,  VA  also  plans  to  compare  the 
new  appeals  process  to  the  current  appeals  process  using  various  performance  metrics, 
such  as  days  to  resolve  an  appeal. 

As  previously  stated  to  GAO,  the  Board  is  creating  an  internal  dashboard  to  track 
performance  metrics  and  will  incorporate  metrics  related  to  the  new  process  into  this 
internal  dashboard  following  passage  of  the  legislation.  As  VA  has  also  reported  to 
GAO,  Digital  Service  at  VA  is  initiating  work  on  the  design  and  development  of  Caseflow 
Dashboard,  which  will  include  the  development  of  a  performance  measurement 
framework.  Caseflow  Dashboard  is  anticipated  to  report  on  data  from  the  entirety  of  the 
appeals  process.  The  Board  already  tracks  internal  performance  metrics,  which  it 
reports  monthly  via  the  Department’s  Monthly  Management  Review.  Analogous 
performance  metrics  related  to  the  new  process  will  be  incorporated  following  passage 
of  the  legislation.  For  example,  the  Board  already  tracks  average  days  to  complete 
original  appeals  and  average  days  to  complete  returned  remands.  The  Board  will 
continue  to  track  these  metrics  for  legacy  appeals,  but,  once  the  new  process  is 
implemented,  can  add  measures  to  track  average  days  to  complete  non-hearing  option 
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Enclosure 


Department  of  Veterans  Affairs  (VA)  Comments  to 
Government  Accountability  Office  (GAO)  Draft  Report 

“VA  DISABILITY  BENEFITS:  Additional  Planning  Would  Enhance  Efforts  to 
Improve  the  Timeliness  of  Appeals  Decisions” 

(GAO-17-234) 

lane  appeals  and  hearing  option  lane  appeals.  The  Board  will  also  continue  to  report 
statistical  data  in  its  Annual  Report  to  Congress,  and  will  incorporate  data  specific  to  the 
new  appeals  process. 

As  previously  reported  to  GAO,  VBA  already  uses  an  extensive  internal  dashboard  to 
track  performance  metrics.  VBA  has  a  fully  integrated  Performance  Analysis  and 
Integrity  Office  that  tracks  and  reports  performance  data  internally  and  externally  daily, 
monthly,  annually,  and  ad-hoc.  In  addition,  VBA’s  Office  of  Field  Operations, 

Operations  Analysis  Staff,  has  a  dedicated  data  analytics  team  that  tracks  performance 
data  for  a  variety  of  purposes,  to  include  trend  analysis  and  predictive  modeling.  Once 
the  appeals  reform  legislation  goes  into  effect  following  the  proposed  18-month 
implementation  period,  VBA  will  track  and  report  on  the  metrics  related  to  the  higher- 
level  review  and  supplemental  claim  lanes  in  the  new  process.  Many  of  the  metrics  will 
be  related  to  claims  processing  timeliness,  pending  inventory,  number  of  completed 
claims,  accuracy,  and  customer  satisfaction. 

Furthermore,  as  indicated,  VA  will  capture  individual  lane  processing  timeliness  and 
customer  satisfaction  data  in  the  new  process  and  compare  it  to  the  processing 
timeliness  of  decisions  and  customer  satisfaction  in  the  legacy  system.  Upon 
enactment  of  the  legislation  and  contingent  upon  appeals  funding  in  VA’s  annual  budget 
appropriation,  at  that  time,  VA  could  develop  a  more  specific  strategy  to  assess  and 
report  on  the  progress  of  the  new  appeals  process,  relative  to  the  current  process. 

Since  a  more  detailed  strategy  is  highly  dependent  on  the  enactment  of  the  appeals 
reform  legislation,  VA  considers  this  recommendation  complete  and  requests  closure. 
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Appendix  II:  GAO  Contact  and  Staff 
Acknowledgments 


GAO  Contact 

Daniel  Bertoni,  Director,  (202)  512-7215  or  bertonid@gao.gov 

Staff 

Acknowledgments 

In  addition  to  the  contact  named  above,  Michele  Grgich  (Assistant 
Director),  Melissa  Jaynes  (Analyst-in-Charge),  Daniel  Concepcion,  and 
Greg  Whitney  made  key  contributions  to  this  report.  Other  key 
contributors  to  this  report  include  James  Bennett,  Mark  Bird,  David 
Chrisinger,  Clifton  Douglas,  Alex  Galuten,  Mitch  Karpman,  Sheila  R. 
McCoy,  Claudine  Pauselli,  Almeta  Spencer,  Eric  Trout,  Walter  Vance, 
and  Tom  Williams. 
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Related  GAO  and  Other  Products 


For  the  purpose  of  evaluating  VA’s  efforts  to  improve  its  appeal 
processing,  we  identified  best  practices  and  other  criteria  related  to 
staffing,  process  reform,  and  IT  upgrades  identified  in  prior  GAO  products 
and  other  publications.  These  included  government-wide  internal  control 
standards;  key  principles  for  effective  strategic  workforce  planning; 
business  process  reengineering  (or  redesign)  best  practices,  and 
information  technology  planning  principles.  We  also  reviewed  additional 
guidance  on  project  management. 

GAO  Products 

Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules. 
GAO-1 6-89G.  Washington,  D.C.:  December  2015. 

Veterans  Benefits  Management  System:  Ongoing  Development  and 
Implementation  Can  Be  Improved;  Goals  are  Needed  to  Promote 
Increased  User  Satisfaction.  GAO-15-582.  Washington,  D.C.:  September 
1,  2015. 

Standards  for  Internal  Control  in  the  Federal  Government.  GAO-1 4-704G. 
Washington,  D.C.:  September  2014. 

Human  Capital:  Strategies  to  Help  Agencies  Meet  Their  Missions  in  an 
Era  of  Highly  Constrained  Resources.  GAO-14-168.  Washington,  D.C.: 
May  7,  2014. 

Managing  For  Results:  Agencies  Should  More  Fully  Develop  Priority 
Goals  under  the  GPRA  Modernization  Act.  GAO-13-174.  Washington, 
D.C.:  April  19,  2013. 

GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for 
Developing  and  Managing  Capital  Program  Costs.  GAO-09-3SP. 
Washington  D.C.:  March  2009. 

Human  Capital:  A  Guide  for  Assessing  Strategic  Training  and 
Development  Efforts  in  the  Federal  Government.  GAO-04-546G. 
Washington,  D.C.:  March  2004. 

Information  Technology  Investment  Management:  A  Framework  for 
Assessing  and  Improving  Process  Maturity.  GAO-04-394G.  Washington, 
D.C.:  March  2004. 
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Combating  Terrorism:  Evaluation  of  Selected  Characteristics  in  National 
Strategies  Related  to  Terrorism.  GAO-04-408T.  Washington,  D.C.: 
February  3,  2004. 

Human  Capital:  Key  Principles  for  Effective  Strategic  Workforce  Planning. 
GAO-04-39.  Washington,  D.C.:  December  11,  2003. 

Human  Capital:  A  Self-Assessment  Checklist  for  Agency  Leaders. 
GAO/OCG-00-14G.  Washington,  D.C.:  September  2000. 

The  Results  Act:  An  Evaluator’s  Guide  to  Assessing  Agency  Annual 
Performance  Plans.  GAO/GGD-10.1.20.  Washington,  D.C.:  April  1998. 

Business  Process  Reengineering  Assessment  Guide. 

GAO/AIMD-10.1 .15.  Washington,  D.C.:  May  1997. 

Executive  Guide:  Effectively  Implementing  the  Government  Performance 
and  Results  Act.  GAO/GGD-96-1 18.  Washington,  D.C.:  June  1996. 

Non-GAO  Publications 

Blackburn,  Simon,  Sarah  Ryerson,  Leigh  Weiss,  Sarah  Wilson,  and 
Carter  Wood,  Insights  into  Organization:  How  Do  I  Implement  Complex 
Change  at  Scale?  Dallas,  Texas:  McKinsey  &  Company,  May  201 1 . 

George,  Michael  L,  David  Rowlands,  Mark  Price,  and  John  Maxey.  The 
Lean  Six  Sigma  Pocket  Toolbook:  A  Quick  Reference  Guide  to  Nearly 
100  Tools  for  Improving  Process  Quality,  Speed,  and  Complexity.  New 
York:  McGraw-Hill,  2005. 

Office  of  Management  and  Budget.  Circular  No.  A-11,  Preparation, 
Submission,  and  Execution  of  the  Budget.  Washington,  D.C.:  July  1, 

2016. 

Project  Management  Institute,  Inc.  A  Guide  To  The  Project  Management 
Body  Of  Knowledge  (PMBOK  Guide)  Newtown  Square,  PA.:  2013. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and  investigative 
arm  of  Congress,  exists  to  support  Congress  in  meeting  its  constitutional 
responsibilities  and  to  help  improve  the  performance  and  accountability  of  the 
federal  government  for  the  American  people.  GAO  examines  the  use  of  public 
funds;  evaluates  federal  programs  and  policies;  and  provides  analyses, 
recommendations,  and  other  assistance  to  help  Congress  make  informed 
oversight,  policy,  and  funding  decisions.  GAO’s  commitment  to  good  government 
is  reflected  in  its  core  values  of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  GAO’s  website  (http://www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  website  newly  released  reports,  testimony,  and  correspondence.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products,  go  to  http://www.gao.gov 
and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of  production  and 
distribution  and  depends  on  the  number  of  pages  in  the  publication  and  whether 
the  publication  is  printed  in  color  or  black  and  white.  Pricing  and  ordering 
information  is  posted  on  GAO’s  website,  http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card,  MasterCard, 

Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Linkedln,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 

Visit  GAO  on  the  web  at  www.gao.gov  and  read  The  Watchblog. 

To  Report  Fraud, 

Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512-4400, 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125, 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 

Washington,  DC  20548 

Strategic  Planning  and 
External  Liaison 

James-Christian  Blockwood,  Managing  Director,  spel@gao.gov,  (202)  512-4707 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7814, 
Washington,  DC  20548 
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